From 9abbaffa228e937df4a8729278593b542766d510 Mon Sep 17 00:00:00 2001 From: Kuan Fan <31664961+kuanfandevops@users.noreply.github.com> Date: Wed, 31 Jan 2024 13:41:48 -0800 Subject: [PATCH] Tracking pull request to merge main-release-jan-2024 to master (#2750) * chore: updated django backend image to newwer version to fix archive dependancy * fix: adjust credit transfer agreement date error message (#2469) Improve the error message shown when a user tries to propose a credit transfer with an empty Agreement Date field. Previous message: 'Error! Date has wrong format. Use one of these formats instead: YYYY-MM-DD.' New message: 'Error! Please enter a valid date in the Agreement Date field: YYYY-MM-DD.' * fix: adjust column widths in xls transactions sheet Reduce the width of the 'Effective Date' column and widen the 'Comments' column in Excel export for better data readability. Due to the previously added 'Category' column, adjust the numbers of all columns after column 7 by increasing them by one. (Column numbers start from 0) * fix: removed 2023 option from compliance report dropdown * fix: remove dropdown options 2023 and later * fix: remove 2023 >= option on dropdown * fix: revert org status label changes on edit organization page This commit reverts unintended label changes on the 'Edit Organization' page to their original state, ensuring UI consistency for IDIR users. Closes #2704 * Bump django from 3.2.20 to 3.2.23 in /backend Bumps [django](https://github.com/django/django) from 3.2.20 to 3.2.23. - [Commits](https://github.com/django/django/compare/3.2.20...3.2.23) --- updated-dependencies: - dependency-name: django dependency-type: direct:production ... Signed-off-by: dependabot[bot] * Compliance Report slow loading time fix Compliance Report Model changes to include latest_report and root_report information To migrate the data to these new columns Using select_related and prefetch_related wherever necessary Making use of the new columns in data retrieval * review comments fixes * remove cache logic Since cache wasn't showing the updated details. have removed it for now. * Exclusion Report not visible to IDIR users fix - Added cache for Compliance Report & Organizations - changes to ComplianceReport Query to retrieve latest reports * feat: updated maxCredits service method to account for reserved credits * fix: updated cache_key to sanitize spaces * bug and failed test cases fix * revert unintended squash change * WIP * Unit test in progress * compliance units update * Compliance Unit changes to compliance report spreadsheet * add pr build template * update * add new pipeline * trigger build * update to use env * update parameters * add brancj build * get branch name * pass branch name * add frontend build * correct frontend build * add test * update version * Build frontend and backend for Jan pipeline (#2421) * Update from 2.7.0 to 3.0.0 * Update Jan pipeline (#2433) * Update frontend route.yaml * update frontend host * update frontend host remove dep * update frontend host name * open build * open deploy dep * Replace Credits and Debits with Compliance Units in Compliance Report Summary View * add for supplemental reports * added backend changes * code enhancement * january 2024 label updates, lint fixes, snapshot updates * director acceptance * initial * permission changes with migration * director acceptance * spreadsheet changes to compliance report as per new act * remove print lines * remove print lines * remove unwanted changes * Add pull request build for Jan release (#2463) * fix defects * update readme * WIP * adding schedule B row add fix * removing consol.log statment * Bug fix for the draft compliance unit * Place compliance units in reserve when submitting a 2023 compliance report with a negative net balance * updating check for a year * Part 2 Summary not populating values from Schedule B/C fixes to 2507 Part 2 Summary not populating values from Schedule B/C fixes to 2507 * Update ScheduleSummaryContainer * System converts credits to compliance units when accepting a complicance report prior to 2023 * Revert "System converts credits to compliance units when accepting a complicance report prior to 2023" This reverts commit f78af66ea51c69a39d42b7e85eb3a74368c383fe. * feat: only show active orgs in credit transfer selection * Compliance unit bug fixes and test cases * minor fixes * Line 28 comments fix * calculation error fix * test scenarios * adding test cases for compliance units * feat: updated organization label column for status to registered * labels for organization edit, details, and create views * feat: added more flexible filtering to org registered column * Prevent Inactive organization from transferring credits buy or sell * System converting credits to compliance units for 2023 prior compliance reports * feat: new credit transaction type added to system administrative adjustment * fix: unit test updates * fixed migration order * fix: missing id * fix: negative frontend validation * rebase fix * UI changes * delta changes fix * minor UI fixes * Compliance Report Spreadsheet fix for date in summary section * Compliance Unit Edge test cases for backend * comment change * include migration for test cases * name conventions fix * supplemntal bug fix for compliance units reporting * chore: merged migrations from rebase * fix: get summary test fix * Remove Actions and Last Transaction column from Organizations table (#2580) * Remove Actions and Last Transaction column from Organizations table * backend change to include/exclude actin from xls * Report History section grouping status changes incorrectly -fix (#2582) Co-authored-by: Prashanth Venkateshappa <54526153+protonater@users.noreply.github.com> * Feat: Label updates for Part 3 Awards, File Submissions - 2491 2492 (#2587) * feat: updated labels and nomenclature for part3 awards and file submissions * chore: linting, snapshots, unit test fixes --------- Co-authored-by: Your Name * fix: refactored get summary method and fixed scenario * Bug for reserve compliance units when submitted. * connect to crunchydb (#2608) * fix for Summary section of compliance units to show changes * minor fix * fix * code optimization * comments for reference * code optimize * optimize * minor fixes * penalty miscalculation fix * status fix revert * adding fix for routing issue of credit/debit * File Submission tab label changes * fix: added missing get_deltas method to update compliance report method * chore: linting, snapshots, test updates * Compliance Unit changes bug fixes for all the scenarios covered. * scenario 17 fix * remove redundant code * fix for TFRS - Scenario 10 not showing the correct math for calculating the penalty amount#2607 * scenario 8 fix * code optimize * TFRS new act label revisions - removing referenes to credits 2583 * fix for Analyst recommend transfer approval buttons not appearing in Jan release * feat: unit test for zero scenario 3 * fix for Inaccurate compliant non-compliant determination in the assessment section for 2023 reporting * Remove/hide balance change in Assessment Section * feat: updated tooltip for compliance units * feat: check for zero balance on director acceptance * fix: unit test update * Feat: Part 3 Award Label updates 2603 * feat: lables updated according to spec * chore: linting fixes and snapshots * Feat: Dashboard Label updates 2474 * Adminstrative Adjustment changes 2431 2583 * chore: updated snapshots and linting * chore: snapshots updated * bug fix for non compliance penalty issue * code optimize * minor fixes * minor fix * fix: formatting fix on react-markdown * New Act Label Changes - Transactions view page using HDE - time based#2602 * TFRS - Summary & Declaration page not loading for 2022 and prior compliance reports#2647 * chore: fixed unit tests for older cases * Updates and fixes for bugs found during testing * label updates * fix: updated logic to prevent zero transactions * New Act labels - transaction status change based on time * TFRS - Time-based transaction label changes for the new Act#2601 * New act label changes - Transactions Page 2495 * fix: label updates for varied views * TFRS - Separate comment disclosure notice based on feature and comment type#2652 * Update langEnUs.js * TFRS - New Act Label Changes - Transactions view page using HDE - time based#2602 * fix: logic fix on less than zero transactions * TFRS - Remove credit market link from BCeID and IDIR dashboards in January release#2661 * fix: label fixes on new initiative agreement, linting, unit tests * fixing wrong comment * Fix: Filtering issues fix for Credit transactions and Compliance Reports * TFRS - File submission filtering from dashboard link not working - IDIR only #2664 * fix: updated transfer labels for BCeID users according to new Act This commit makes the following updates to the New Transfer page for BCeID users, ensuring alignment with the new Act: - Change the dropdown label from 'Select a Fuel Supplier' to 'Select an Organization'. - The signing authority declaration statement is now updated to: 'I confirm that records evidencing each matter reported under section 17 of the Low Carbon Fuel (General) Regulation are available on request.' Additionally, a database migration is added to apply the signing authority declaration statement label change within the existing record. Closes #2690 * refactor: changed error logging to warning and added raise in migration * add jan release test deploy * specify branch name for jab test deploy * add test deploy * add runs-on * add runs-on * add branches for workflow_dispatch * add new value files for jan dry run * deploy jan release to test * split test approval and deploy * fix: rearranged migration order to sync with main release branch * fix: rearranged migration order to match master * fix: updated migration order to match master for upcoming merge (#2742) * update pipeline merge install and upgrade * add notification sever * update notification server * update notification serverimage name * update notification certs url * add keycloak url and adjust resource * update knps for jan release * update existing network policy * fix: update labels and remove old features on add/edit organization * chore: reverted year limit on compliance report * refactor: lint fix, removed semicolons, switched to single quotes * refactor: lint fix, removed extra commas * fix: renamed label name following linting adjustments * fix: updated organization add/edit form test snapshot * fix: updated labels * fix: fixed non status related loading for file submissions page * fix: revise error message for insufficient compliance units * fix: fix wrong labels in transfer confirmation modal * fix: fix broken file submission link on BCeID dashboard * fix: TFRS - Update Notification descriptions to align with new Act labels #2765 * fix: hide lcfs email on org details page for idir users * add jab release drawino and upgrade autoscaling apiVersion * update char number * test workflow * disable auto scaling for dev * update autoscaling version * update teh workflow to fast deployment * open the test and linting * feat: TFRS - Adjust 'Part 3 Award' label for transaction type in the Historical Data Entry feature #2773 * fix: fix incorrect labels in initiative agreement issuance transaction * fix: update labels in notifications settings as per new act * fix: update bceid and idir guide links to external urls * fix: correct labels in transfer confirmation modal * fix: correct effective date of transfer 2095 * HDE Transactions efective date fix in spreadsheet * fix: fixes on label and compliance years * fix: add a reverse noop to avoid downgrade issues * fix part 3 label in the historical data entry table * fix: update two notification lables * fix: update modal confirmLabel logic to handle different transaction types * fix: fix compliance period year of transfer 2095 --------- Signed-off-by: dependabot[bot] Co-authored-by: Your Name Co-authored-by: Alex Zorkin <47334977+AlexZorkin@users.noreply.github.com> Co-authored-by: Hamed Valiollahi Bayeki Co-authored-by: Kevin Hashimoto Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Prashanth Venkateshappa <54526153+protonater@users.noreply.github.com> Co-authored-by: Prashanth Co-authored-by: jig-patel <122304104+jig-patel@users.noreply.github.com> Co-authored-by: Prashanth <130073308+prv-proton@users.noreply.github.com> --- .github/readme.md | 11 +- .github/workflows/branch-deploy-template.yaml | 142 ++ .github/workflows/build-template.yaml | 233 +++ .github/workflows/dev-test-jan-release.yaml | 130 ++ .github/workflows/pr-dev-cicd.yaml | 54 + .../workflows/pr-dev-database-template.yaml | 69 + .github/workflows/pr-dev-deploy-template.yaml | 191 ++ .github/workflows/pr-teardown.yaml | 41 + README.md | 1 + .../test/test_carbon_intensity_limits.json | 39 +- .../api/fixtures/test/test_fuel_codes.json | 24 + .../test_post_compliance_unit_reporting.json | 378 ++++ .../test_pre_compliance_unit_reporting.json | 378 ++++ .../api/migrations/0014_new_label_changes.py | 216 +++ ...0015_add_admin_adjustment_20230808_1803.py | 34 + ...016_alter_credittrade_number_of_credits.py | 18 + ...7_alter_compliace_report_history_status.py | 20 + .../0018_report_history_grouping.py | 58 + ...signing_authority_declaration_statement.py | 38 + ...correct_effective_date_of_transfer_2095.py | 39 + ...rect_compliance_period_of_transfer_2095.py | 40 + backend/api/models/ComplianceReport.py | 3 +- backend/api/models/CreditTrade.py | 9 +- backend/api/models/CreditTradeType.py | 22 + backend/api/serializers/ComplianceReport.py | 318 ++-- backend/api/serializers/CreditTrade.py | 13 +- backend/api/serializers/CreditTradeType.py | 2 +- backend/api/serializers/Document.py | 24 +- .../api/services/ComplianceReportService.py | 81 +- .../services/ComplianceReportSpreadSheet.py | 59 +- .../ComplianceReportSummaryService.py | 262 +++ backend/api/services/CreditTradeService.py | 18 + backend/api/services/OrganizationService.py | 131 +- backend/api/services/SpreadSheetBuilder.py | 30 +- backend/api/tests/base_test_case.py | 5 +- .../payloads/compliance_unit_payloads.py | 156 ++ .../test_compliance_supplemental_reporting.py | 2 +- ...st_compliance_unit_reporting_after_2023.py | 1016 +++++++++++ ...t_compliance_unit_reporting_before_2023.py | 849 +++++++++ .../test_credit_trade_admin_adjustment.py | 142 ++ backend/api/tests/test_get_summary.py | 3 + backend/api/validators.py | 8 +- backend/api/viewsets/ComplianceReport.py | 119 +- backend/api/viewsets/CreditTrade.py | 28 +- backend/api/viewsets/Organization.py | 26 +- backend/tfrs/urls.py | 2 +- charts/tfrs-apps/Chart.yaml | 4 +- .../tfrs-apps/charts/tfrs-backend/Chart.yaml | 2 +- .../tfrs-backend/templates/_helpers.tpl | 80 +- .../templates/deployment-config.yaml | 25 +- .../charts/tfrs-backend/templates/hpa.yaml | 32 + .../charts/tfrs-backend/templates/route.yaml | 20 + .../tfrs-backend/templates/service.yaml | 15 + .../charts/tfrs-backend/values-dev-jan.yaml | 29 +- .../charts/tfrs-backend/values-test-jan.yaml | 34 + .../tfrs-apps/charts/tfrs-backend/values.yaml | 82 - .../tfrs-apps/charts/tfrs-celery/.helmignore | 23 + .../tfrs-apps/charts/tfrs-celery/Chart.yaml | 26 + .../charts/tfrs-celery/templates/_helpers.tpl | 66 + .../templates/deployment-config.yaml | 104 ++ .../charts/tfrs-celery/values-dev-jan.yaml | 19 + .../charts/tfrs-celery/values-test-jan.yaml | 19 + .../charts/tfrs-frontend/.helmignore | 23 + .../tfrs-apps/charts/tfrs-frontend/Chart.yaml | 24 + .../tfrs-frontend/templates/_helpers.tpl | 66 + .../tfrs-frontend/templates/configmap.yaml | 28 + .../templates/deployment-config.yaml | 95 + .../charts/tfrs-frontend/templates/hpa.yaml | 32 + .../charts/tfrs-frontend/templates/route.yaml | 22 + .../tfrs-frontend/templates/service.yaml | 16 + .../charts/tfrs-frontend/values-dev-jan.yaml | 28 + .../charts/tfrs-frontend/values-test-jan.yaml | 28 + .../tfrs-notification-server/.helmignore | 23 + .../tfrs-notification-server/Chart.yaml | 24 + .../templates/_helpers.tpl | 67 + .../templates/deployment-config.yaml | 93 + .../templates/hpa.yaml | 32 + .../templates/route.yaml | 22 + .../templates/service.yaml | 17 + .../values-dev-jan.yaml | 25 + .../values-test-jan.yaml | 25 + .../charts/tfrs-scan-coordinator/.helmignore | 23 + .../charts/tfrs-scan-coordinator/Chart.yaml | 24 + .../templates/_helpers.tpl | 66 + .../templates/deployment-config.yaml | 83 + .../tfrs-scan-coordinator/values-dev-jan.yaml | 12 + .../values-test-jan.yaml | 12 + .../charts/tfrs-scan-handler/.helmignore | 23 + .../charts/tfrs-scan-handler/Chart.yaml | 24 + .../tfrs-scan-handler/templates/_helpers.tpl | 62 + .../templates/deployment-config.yaml | 82 + .../tfrs-scan-handler/values-dev-jan.yaml | 12 + .../tfrs-scan-handler/values-test-jan.yaml | 12 + charts/tfrs-spilo/values-dev.yaml | 2 +- .../getCreditTransferType.js | 40 +- .../prepareCreditTransfer.js | 80 +- frontend/package.json | 2 +- .../src/actions/creditTransfersActions.js | 20 +- frontend/src/actions/organizationActions.js | 31 +- .../components/HistoricalConfirmationTable.js | 131 ++ .../HistoricalDataEntryFormDetails.js | 15 +- .../components/HistoricalDataEntryPage.js | 32 +- .../components/HistoricalDataTable.js | 4 +- frontend/src/app/components/Navbar.js | 24 +- frontend/src/app/components/Tooltip.js | 2 +- .../ComplianceReportingEditContainer.js | 2 +- .../ScheduleAssessmentContainer.js | 15 +- .../ScheduleBContainer.js | 158 +- .../ScheduleDContainer.js | 27 +- .../ScheduleSummaryContainer.js | 41 +- .../ScheduleSummaryContainer.test.js | 6 + .../ScheduleSummaryContainer.test.js.snap | 12 +- .../components/ScheduleBContainer.test.js | 930 ++++++++++ .../components/ScheduleBTotals.test.js | 386 ++++ .../ScheduleBContainer.test.js.snap | 1562 +++++++++++++++++ .../ScheduleBTotals.test.js.snap | 85 + .../components/Columns.js | 36 +- .../components/ComplianceReportingPage.js | 2 +- .../components/ReportHistory.js | 2 +- .../components/ScheduleAssessmentPage.js | 12 +- .../components/ScheduleBTotals.js | 24 +- .../components/ScheduleDTabs.js | 12 +- .../components/ScheduleDeltas.js | 144 +- .../components/SchedulesPage.js | 5 +- .../components/SnapshotDisplay.js | 452 ++++- .../components/SummaryLCFSDeltas.js | 252 +++ .../constants/actionTypes/Organizations.js | 2 + frontend/src/constants/langEnUs.js | 27 +- frontend/src/constants/notificationTypes.js | 24 +- .../constants/reducerTypes/Organizations.js | 5 +- .../constants/schedules/scheduleColumns.js | 10 +- .../settings/notificationsCreditTransfers.js | 14 +- .../notificationsGovernmentTransfers.js | 8 +- frontend/src/constants/tooltips.js | 5 + frontend/src/constants/values.js | 13 +- .../CreditTransactionsContainer.js | 2 +- .../CreditTransferAddContainer.js | 11 +- .../CreditTransferEditContainer.js | 9 +- .../CreditTransferViewContainer.js | 121 +- .../CreditTransferSigningHistory.test.js | 22 +- .../CreditTransferSigningHistory.test.js.snap | 35 +- .../components/CreditTransactionsPage.js | 15 +- .../components/CreditTransferComment.js | 6 +- .../components/CreditTransferCommentForm.js | 2 +- .../components/CreditTransferDetails.js | 33 +- .../components/CreditTransferForm.js | 34 +- .../components/CreditTransferFormButtons.js | 21 +- .../components/CreditTransferFormDetails.js | 22 +- .../components/CreditTransferProgress.js | 13 +- .../CreditTransferSigningHistory.js | 59 +- .../components/CreditTransferTable.js | 45 +- .../components/CreditTransferTerms.js | 2 +- .../CreditTransferTextRepresentation.js | 137 +- .../CreditTransferVisualRepresentation.js | 48 +- .../components/GovernmentTransferForm.js | 7 +- .../GovernmentTransferFormDetails.js | 10 +- .../components/ModalSubmitCreditTransfer.js | 14 +- .../NewInitiativeagreementComment.js | 165 ++ .../dashboard/components/Administration.js | 4 +- frontend/src/dashboard/components/Balance.js | 11 +- .../src/dashboard/components/BalanceBCEID.js | 14 +- .../dashboard/components/ComplianceReports.js | 4 +- .../components/ComplianceReportsBCEID.js | 6 +- .../components/CreditTradingValue.js | 30 - .../components/CreditTransactions.js | 32 +- .../components/CreditTransactionsBCEID.js | 14 +- .../src/dashboard/components/DashboardPage.js | 19 +- .../dashboard/components/DirectorReview.js | 15 +- .../{Part3Agreements.js => FileSubmission.js} | 39 +- .../dashboard/components/FileSubmissions.js | 4 +- .../src/dashboard/components/FuelCodes.js | 2 +- .../dashboard/components/LcfsWebsiteLink.js | 25 + .../components/OrganizationDetails.js | 25 +- .../src/dashboard/components/UserSettings.js | 6 +- .../ComplianceReports.test.js.snap | 8 +- .../ComplianceReportsBCEID.test.js.snap | 12 +- .../__snapshots__/DirectorReview.test.js.snap | 6 +- .../ExclusionAgreementContainer.js | 1 - .../components/NotificationsTable.js | 2 +- .../organizations/OrganizationsContainer.js | 2 +- .../components/OrganizationDetails.js | 34 +- .../components/OrganizationEditForm.js | 123 +- .../components/OrganizationsPage.js | 2 +- .../components/OrganizationsTable.js | 49 +- .../__tests__/OrganizationEditForm.test.js | 4 +- .../OrganizationEditForm.test.js.snap | 31 +- frontend/src/reducers/organizationReducer.js | 24 +- frontend/src/reducers/reducer.js | 3 +- .../schedule_summary/Part3SummaryContainer.js | 247 ++- .../schedule_summary/ScheduleSummaryPart3.js | 200 ++- .../Part3/Part3CalculatePart3.test.js | 14 +- .../__tests__/Part3/Part3LineData.test.js | 27 +- .../Part3/Part3SummaryContainer.test.js | 12 + .../__tests__/Part3/Part3TableData.test.js | 4 +- .../Part3/Part3TooltipCarbonPenalty.test.js | 4 +- .../SecureFileSubmissionAddContainer.js | 12 +- .../SecureFileSubmissionContainer.js | 16 +- .../SecureFileSubmissionDetailContainer.js | 2 +- .../SecureFileSubmissionEditContainer.js | 8 +- .../LinkedCreditTransferSelection.js | 16 +- .../components/SecureFileSubmissionDetails.js | 11 +- .../SecureFileSubmissionFileAttachments.js | 2 +- .../components/SecureFileSubmissionForm.js | 3 +- .../SecureFileSubmissionFormDetails.js | 29 +- .../components/SecureFileSubmissionTable.js | 11 +- .../components/SecureFileSubmissionsPage.js | 25 +- .../settings/components/SettingsDetails.js | 4 +- .../src/settings/components/SettingsTabs.js | 2 +- frontend/src/utils/functions.js | 69 +- frontend/src/utils/toastr.js | 21 +- frontend/styles/CreditTransfers.scss | 2 +- frontend/styles/HistoricalDataEntry.scss | 11 + frontend/styles/SecureDocumentUpload.scss | 26 +- .../groovy/pages/app/OrganizationsPage.groovy | 2 +- openshift-v4/templates/celery/Dockerfile | 14 + .../templates/celery/celery-bc-docker.yaml | 89 + .../knp/2-allow-backend-accepts.yaml | 32 + .../templates/knp/2-allow-clamav-accepts.yaml | 39 + .../templates/knp/2-allow-minio-accepts.yaml | 46 + .../knp/2-allow-rabbitmq-accepts.yaml | 73 + .../templates/knp/2-allow-spilo-accepts.yaml | 97 + openshift-v4/templates/knp/2-apps.yaml | 63 - openshift-v4/templates/knp/3-spilo.yaml | 106 -- .../templates/knp/4-clamav-rabbitmq.yaml | 102 -- .../templates/knp/TFRS-Jan-release.drawio | 86 + .../jan-allow-backend-accepts-test.yaml | 23 + .../jan-allow-clamav-accepts-test.yaml | 30 + .../jan-allow-crunchy-accepts-test.yaml | 86 + .../jan-allow-minio-accepts-test.yaml | 37 + .../jan-allow-rabbitmq-accepts-test.yaml | 65 + .../templates/knp/knp-diagram-2.0.0.drawio | 221 ++- .../templates/scan-handler/Dockerfile | 14 + .../scan-handler/scan-handler-bc-docker.yaml | 89 + scripts/import-data.sh | 6 +- 234 files changed, 13705 insertions(+), 1620 deletions(-) create mode 100644 .github/workflows/branch-deploy-template.yaml create mode 100644 .github/workflows/build-template.yaml create mode 100644 .github/workflows/dev-test-jan-release.yaml create mode 100644 .github/workflows/pr-dev-cicd.yaml create mode 100644 .github/workflows/pr-dev-database-template.yaml create mode 100644 .github/workflows/pr-dev-deploy-template.yaml create mode 100644 .github/workflows/pr-teardown.yaml create mode 100644 backend/api/fixtures/test/test_post_compliance_unit_reporting.json create mode 100644 backend/api/fixtures/test/test_pre_compliance_unit_reporting.json create mode 100644 backend/api/migrations/0014_new_label_changes.py create mode 100644 backend/api/migrations/0015_add_admin_adjustment_20230808_1803.py create mode 100644 backend/api/migrations/0016_alter_credittrade_number_of_credits.py create mode 100644 backend/api/migrations/0017_alter_compliace_report_history_status.py create mode 100644 backend/api/migrations/0018_report_history_grouping.py create mode 100644 backend/api/migrations/0019_update_signing_authority_declaration_statement.py create mode 100644 backend/api/migrations/0020_correct_effective_date_of_transfer_2095.py create mode 100644 backend/api/migrations/0021_correct_compliance_period_of_transfer_2095.py create mode 100644 backend/api/services/ComplianceReportSummaryService.py create mode 100644 backend/api/tests/payloads/compliance_unit_payloads.py create mode 100644 backend/api/tests/test_compliance_unit_reporting_after_2023.py create mode 100644 backend/api/tests/test_compliance_unit_reporting_before_2023.py create mode 100644 backend/api/tests/test_credit_trade_admin_adjustment.py create mode 100644 charts/tfrs-apps/charts/tfrs-backend/templates/hpa.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-backend/templates/route.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-backend/templates/service.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-backend/values-test-jan.yaml delete mode 100644 charts/tfrs-apps/charts/tfrs-backend/values.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-celery/.helmignore create mode 100644 charts/tfrs-apps/charts/tfrs-celery/Chart.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-celery/templates/_helpers.tpl create mode 100644 charts/tfrs-apps/charts/tfrs-celery/templates/deployment-config.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-celery/values-dev-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-celery/values-test-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/.helmignore create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/Chart.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/templates/_helpers.tpl create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/templates/configmap.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/templates/deployment-config.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/templates/hpa.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/templates/route.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/templates/service.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/values-dev-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-frontend/values-test-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/.helmignore create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/Chart.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/templates/_helpers.tpl create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/templates/deployment-config.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/templates/hpa.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/templates/route.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/templates/service.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/values-dev-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-notification-server/values-test-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-scan-coordinator/.helmignore create mode 100644 charts/tfrs-apps/charts/tfrs-scan-coordinator/Chart.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-scan-coordinator/templates/_helpers.tpl create mode 100644 charts/tfrs-apps/charts/tfrs-scan-coordinator/templates/deployment-config.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-scan-coordinator/values-dev-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-scan-coordinator/values-test-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-scan-handler/.helmignore create mode 100644 charts/tfrs-apps/charts/tfrs-scan-handler/Chart.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-scan-handler/templates/_helpers.tpl create mode 100644 charts/tfrs-apps/charts/tfrs-scan-handler/templates/deployment-config.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-scan-handler/values-dev-jan.yaml create mode 100644 charts/tfrs-apps/charts/tfrs-scan-handler/values-test-jan.yaml create mode 100644 frontend/src/admin/historical_data_entry/components/HistoricalConfirmationTable.js create mode 100644 frontend/src/compliance_reporting/__tests__/components/ScheduleBContainer.test.js create mode 100644 frontend/src/compliance_reporting/__tests__/components/ScheduleBTotals.test.js create mode 100644 frontend/src/compliance_reporting/__tests__/components/__snapshots__/ScheduleBContainer.test.js.snap create mode 100644 frontend/src/compliance_reporting/__tests__/components/__snapshots__/ScheduleBTotals.test.js.snap create mode 100644 frontend/src/compliance_reporting/components/SummaryLCFSDeltas.js create mode 100644 frontend/src/constants/tooltips.js create mode 100644 frontend/src/credit_transfers/components/NewInitiativeagreementComment.js delete mode 100644 frontend/src/dashboard/components/CreditTradingValue.js rename frontend/src/dashboard/components/{Part3Agreements.js => FileSubmission.js} (50%) create mode 100644 frontend/src/dashboard/components/LcfsWebsiteLink.js create mode 100644 openshift-v4/templates/celery/Dockerfile create mode 100644 openshift-v4/templates/celery/celery-bc-docker.yaml create mode 100644 openshift-v4/templates/knp/2-allow-backend-accepts.yaml create mode 100644 openshift-v4/templates/knp/2-allow-clamav-accepts.yaml create mode 100644 openshift-v4/templates/knp/2-allow-minio-accepts.yaml create mode 100644 openshift-v4/templates/knp/2-allow-rabbitmq-accepts.yaml create mode 100644 openshift-v4/templates/knp/2-allow-spilo-accepts.yaml delete mode 100644 openshift-v4/templates/knp/2-apps.yaml delete mode 100644 openshift-v4/templates/knp/3-spilo.yaml delete mode 100644 openshift-v4/templates/knp/4-clamav-rabbitmq.yaml create mode 100644 openshift-v4/templates/knp/TFRS-Jan-release.drawio create mode 100644 openshift-v4/templates/knp/jan-test/jan-allow-backend-accepts-test.yaml create mode 100644 openshift-v4/templates/knp/jan-test/jan-allow-clamav-accepts-test.yaml create mode 100644 openshift-v4/templates/knp/jan-test/jan-allow-crunchy-accepts-test.yaml create mode 100644 openshift-v4/templates/knp/jan-test/jan-allow-minio-accepts-test.yaml create mode 100644 openshift-v4/templates/knp/jan-test/jan-allow-rabbitmq-accepts-test.yaml create mode 100644 openshift-v4/templates/scan-handler/Dockerfile create mode 100644 openshift-v4/templates/scan-handler/scan-handler-bc-docker.yaml diff --git a/.github/readme.md b/.github/readme.md index 76922ff75..170b450f7 100644 --- a/.github/readme.md +++ b/.github/readme.md @@ -40,8 +40,17 @@ * tfrs-release.yaml (TFRS release-2.10.0): the pipeline builds the release and deploys on Test and Prod, it needs to be manually triggered * create-release.yaml (Create Release after merging to master): tag and create the release after merging release branch to master. The description of the tracking pull request becomes release notes +* dev-jan-release.yaml (TFRS Dev Jan Release): the pipeline build Jan 2024 release and deploy on dev for every commit +* dev-release.yaml (TFRS Dev release-2.9.0): the pipeline is automatically triggered when there is a commit to the release branch +* tfrs-release.yaml (TFRS release-2.9.0): the pipelin builds the release and deploy on Test and Prod, it needs to be manually triggered + ## Other Pipelines +* branch-deploy-template.yaml (Branch Deploy Template): a pipeline template to deploy a branch +* build-template.yaml (Build Template): a pipeline template to build branch or pull request * cleanup-cron-workflow-runs.yaml (Scheduled cleanup old workflow runs): a cron job to cleanup the old workflows * cleanup-workflow-runs.yaml (Cleanup old workflow runs): manually cleanup teh workflow runs - +* pr-dev-cicd.yaml (TFRS Dev Jan PR CICD): the pipeline builds Jan 2024 pull requests and deploy on dev if the pull request title ends with build-on-dev +* pr-dev-database-template.yaml (PR Dev Database Template): the template to create database for pull request build +* pr-deploy-template (PR Dev Deploy Template): the template deploys pull request build to dev +* pr-teardown.yaml (TFRS Dev Jan PR Teardown): tear down the Jan 2024 pull request builds from dev diff --git a/.github/workflows/branch-deploy-template.yaml b/.github/workflows/branch-deploy-template.yaml new file mode 100644 index 000000000..a245407b8 --- /dev/null +++ b/.github/workflows/branch-deploy-template.yaml @@ -0,0 +1,142 @@ +name: Branch Deploy Template + +on: + workflow_call: + inputs: + branch-name: # sample value: release-2.9.0 or main-release-jan-2024 + required: true + type: string + # suffix is in format of -dev, -test, -dev-jan, test-jan, -dev-1923, dev-jan-1923 + suffix: + required: true + type: string + # env-name is in format of dev, test + env-name: + required: true + type: string + # database-service-host-name, sample tfrs-spilo, tfrs-spilo-jan, tfrs-spilo-dev-1988 + database-service-host-name: + required: true + type: string + # this virtual host name, sample tfrs-jan-vhost + rabbitmq-vhost: + required: true + type: string + secrets: + tools-namespace: + required: true + namespace: + required: true + openshift-server: + required: true + openshift-token: + required: true + +jobs: + + deploy: + + name: Deploy tfrs + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3 + with: + ref: ${{ inputs.branch-name }} + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.tools-namespace }} + + - name: Deploy tfrs-frontend + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-frontend:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-frontend:${{ inputs.env-name }}-${{ inputs.branch-name }} + cd charts/tfrs-apps/charts/tfrs-frontend + helm status -n ${{ secrets.namespace }} tfrs-frontend${{ inputs.suffix }} + helm upgrade --install \ + --set frontendImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \ + --set suffix=${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=${{ inputs.env-name }} \ + -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-frontend${{ inputs.suffix }} . + + - name: Deploy tfrs-backend + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-backend:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-backend:${{ inputs.env-name }}-${{ inputs.branch-name }} + cd charts/tfrs-apps/charts/tfrs-backend + helm status -n ${{ secrets.namespace }} tfrs-backend${{ inputs.suffix }} + helm upgrade --install \ + --set backendImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \ + --set suffix=${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=${{ inputs.env-Name }} \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \ + -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-backend${{ inputs.suffix }} . + + - name: Deploy tfrs-celery + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-celery:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-celery:${{ inputs.env-name }}-${{ inputs.branch-name }} + cd charts/tfrs-apps/charts/tfrs-celery + helm status -n ${{ secrets.namespace }} tfrs-celery${{ inputs.suffix }} + helm upgrade --install \ + --set celeryImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \ + --set suffix=${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=${{ inputs.env-Name }} \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \ + -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-celery${{ inputs.suffix }} . + + - name: Deploy tfrs-scan-handler + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-scan-handler:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-scan-handler:${{ inputs.env-name }}-${{ inputs.branch-name }} + cd charts/tfrs-apps/charts/tfrs-scan-handler + helm status -n ${{ secrets.namespace }} tfrs-scan-handler${{ inputs.suffix }} + helm upgrade --install \ + --set scanHandlerImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \ + --set suffix=${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=${{ inputs.env-Name }} \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \ + -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-scan-handler${{ inputs.suffix }} . + + - name: Deploy tfrs-scan-coordinator + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-scan-coordinator:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-scan-coordinator:${{ inputs.env-name }}-${{ inputs.branch-name }} + cd charts/tfrs-apps/charts/tfrs-scan-coordinator + helm status -n ${{ secrets.namespace }} tfrs-scan-coordinator${{ inputs.suffix }} + helm upgrade --install \ + --set scanCoordinatorImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \ + --set suffix=${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=${{ inputs.env-Name }} \ + --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \ + -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-scan-coordinator${{ inputs.suffix }} . + + - name: Deploy tfrs-notification-server + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-notification-server:build-${{ inputs.branch-name }} ${{ secrets.namespace }}/tfrs-notification-server:${{ inputs.env-name }}-${{ inputs.branch-name }} + cd charts/tfrs-apps/charts/tfrs-notification-server + helm status -n ${{ secrets.namespace }} tfrs-notification-server${{ inputs.suffix }} + helm upgrade --install \ + --set notificationServerImageTagName=${{ inputs.env-name }}-${{ inputs.branch-name }} \ + --set suffix=${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=${{ inputs.env-Name }} \ + --set rabbitmqVHost=${{ inputs.rabbitmq-vhost }} \ + -n ${{ secrets.namespace }} -f ./values${{ inputs.suffix }}.yaml tfrs-notification-server${{ inputs.suffix }} . \ No newline at end of file diff --git a/.github/workflows/build-template.yaml b/.github/workflows/build-template.yaml new file mode 100644 index 000000000..ed780c316 --- /dev/null +++ b/.github/workflows/build-template.yaml @@ -0,0 +1,233 @@ + +# This template supports both pr build and branch build +name: Build Template + +on: + workflow_call: + inputs: + # when build branch, the sample value is -main-release-jan-2024 + # when build pull request, the sample value is -jan-2024 + suffix: + required: true + type: string + # when build branch, the sample value is main-release-jan-2024 + # when build pull request, the sample value is refs/pull/2024/head + checkout-ref: + required: true + type: string + secrets: + tools-namespace: + required: true + openshift-server: + required: true + openshift-token: + required: true + +env: + GIT_URL: https://github.com/bcgov/tfrs.git + +jobs: + + build-backend: + + name: Build TFRS Backend on Openshift + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3.5.3 + with: + ref: ${{ inputs.checkout-ref }} + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.tools-namespace }} + + - name: Build tfrs Backend + run: | + cd openshift-v4/templates/backend + oc process -f ./backend-bc.yaml NAME=tfrs \ + SUFFIX=-build${{ inputs.suffix}} \ + VERSION=build${{ inputs.suffix }} \ + GIT_URL=${{ env.GIT_URL }} \ + GIT_REF=${{ inputs.checkout-ref }} \ + | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }} + oc cancel-build bc/tfrs-backend-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true + oc start-build --wait=true tfrs-backend-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} + + build-frontend: + + name: Build TFRS Frontend on Openshift + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3.5.3 + with: + ref: ${{ inputs.checkout-ref }} + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.tools-namespace }} + + - name: Build TFRS Frontend + run: | + cd openshift-v4/templates/frontend + oc process -f ./frontend-bc-docker.yaml NAME=tfrs \ + SUFFIX=-build${{ inputs.suffix}} \ + VERSION=build${{ inputs.suffix }} \ + GIT_URL=${{ env.GIT_URL }} \ + GIT_REF=${{ inputs.checkout-ref }} \ + | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }} + oc cancel-build bc/tfrs-frontend-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true + oc start-build --wait=true tfrs-frontend-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} + + build-celery: + + name: Build TFRS Celery on Openshift + needs: [build-frontend, build-backend] + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3.5.3 + with: + ref: ${{ inputs.checkout-ref }} + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.tools-namespace }} + + - name: Build TFRS Celery + run: | + cd openshift-v4/templates/celery + pwd + ls -l + oc process -f ./celery-bc-docker.yaml NAME=tfrs \ + SUFFIX=-build${{ inputs.suffix}} \ + VERSION=build${{ inputs.suffix }} \ + GIT_URL=${{ env.GIT_URL }} \ + GIT_REF=${{ inputs.checkout-ref }} \ + | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }} + oc cancel-build bc/tfrs-celery-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true + oc start-build --wait=true tfrs-celery-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} + + build-scan-coordinator: + + name: Build TFRS Scan Coordinator on Openshift + needs: [build-frontend, build-backend] + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3.5.3 + with: + ref: ${{ inputs.checkout-ref }} + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.tools-namespace }} + + - name: Build TFRS Scan Coordinator + run: | + cd openshift-v4/templates/scan-coordinator + oc process -f ./scan-coordinator-bc.yaml NAME=tfrs \ + SUFFIX=-build${{ inputs.suffix}} \ + VERSION=build${{ inputs.suffix }} \ + GIT_URL=${{ env.GIT_URL }} \ + GIT_REF=${{ inputs.checkout-ref }} \ + | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }} + oc cancel-build bc/tfrs-scan-coordinator-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true + oc start-build --wait=true tfrs-scan-coordinator-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} + + build-scan-handler: + + name: Build TFRS Scan Handler on Openshift + needs: [build-scan-coordinator, build-celery] + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3.5.3 + with: + ref: ${{ inputs.checkout-ref }} + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.tools-namespace }} + + - name: Build TFRS Scan-Handler + run: | + cd openshift-v4/templates/scan-handler + oc process -f ./scan-handler-bc-docker.yaml NAME=tfrs \ + SUFFIX=-build${{ inputs.suffix}} \ + VERSION=build${{ inputs.suffix }} \ + GIT_URL=${{ env.GIT_URL }} \ + GIT_REF=${{ inputs.checkout-ref }} \ + | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }} + oc cancel-build bc/tfrs-scan-handler-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true + oc start-build --wait=true tfrs-scan-handler-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} + + build-notification-server: + + name: Build TFRS Notification Server on Openshift + needs: [build-scan-coordinator, build-celery] + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3.5.3 + with: + ref: ${{ inputs.checkout-ref }} + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.tools-namespace }} + + - name: Build TFRS Notification Server + run: | + cd openshift-v4/templates/notification + oc process -f ./notification-server-bc.yaml NAME=tfrs \ + SUFFIX=-build${{ inputs.suffix}} \ + VERSION=build${{ inputs.suffix }} \ + GIT_URL=${{ env.GIT_URL }} \ + GIT_REF=${{ inputs.checkout-ref }} \ + | oc apply --wait=true -f - -n ${{ secrets.tools-namespace }} + oc cancel-build bc/tfrs-notification-server-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} || true + oc start-build --wait=true tfrs-notification-server-build${{ inputs.suffix}} -n ${{ secrets.tools-namespace }} \ No newline at end of file diff --git a/.github/workflows/dev-test-jan-release.yaml b/.github/workflows/dev-test-jan-release.yaml new file mode 100644 index 000000000..21704738f --- /dev/null +++ b/.github/workflows/dev-test-jan-release.yaml @@ -0,0 +1,130 @@ + +## For each release, the value of name, branches, RELEASE_NAME and PR_NUMBER need to be adjusted accordingly +## For each release, update lib/config.js: version and releaseBranch + +name: TFRS Dev/Test Jan Release + +on: + push: + branches: [ main-release-jan-2024 ] + # paths: + # - frontend/** + # - backend/** + # - security-scan/** + workflow_dispatch: + branches: + - main-release-jan-2024 + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + + unit-test: + + name: Run Backend Unit Tests + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + - name: Checkout + uses: actions/checkout@v3.5.3 + + - name: Run coverage report for django tests + uses: kuanfandevops/django-test-action@itvr-django-test + continue-on-error: true + with: + settings-dir-path: "backend/api" + requirements-file: "backend/requirements.txt" + managepy-dir: backend + + lint: + + name: Linting + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + - name: Checkout + uses: actions/checkout@v3.5.3 + + - name: Frontend Linting + continue-on-error: true + run: | + cd frontend + pwd + npm install + npm run lint + + - name: Backend linting + uses: github/super-linter/slim@v4 + continue-on-error: true + env: + DEFAULT_BRANCH: ${{ github.ref_name }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + FILTER_REGEX_INCLUDE: .*backend/.*.py + VALIDATE_PYTHON_PYLINT: true + LOG_LEVEL: WARN + + # when build branch, the suffix sample is -main-release-jan-2024 + # the checkout-ref sample is main-release-jan-2024 + build: + name: Build + needs: [unit-test, lint] + uses: ./.github/workflows/build-template.yaml + with: + suffix: -${{ github.ref_name }} + checkout-ref: ${{ github.ref_name }} + secrets: + tools-namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-tools + openshift-server: ${{ secrets.OPENSHIFT_SERVER }} + openshift-token: ${{ secrets.OPENSHIFT_TOKEN }} + + # The suffix is -dev-jan, the deployment names are tfrs-backend-dev-jan, tfrs-frontend-dev-jan and etc.. + # The image tags are tfrs-backend:dev-main-release-jan-2024, tfrs-frontend:dev-main-release-jan-2024 and etc.. + deploy-on-dev: + name: Deploy on Dev + needs: build + uses: ./.github/workflows/branch-deploy-template.yaml + with: + branch-name: ${{ github.ref_name }} + suffix: -dev-jan + env-name: dev + database-service-host-name: tfrs-crunchy-dev-pgbouncer + rabbitmq-vhost: tfrs-jan-vhost + secrets: + tools-namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-tools + namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-dev + openshift-server: ${{ secrets.OPENSHIFT_SERVER }} + openshift-token: ${{ secrets.OPENSHIFT_TOKEN }} + + approval-test-deployment: + name: Approval deployment on Test + runs-on: ubuntu-latest + needs: deploy-on-dev + timeout-minutes: 60 + steps: + - name: Ask for approval for TFRS Test deployment + uses: trstringer/manual-approval@v1.6.0 + with: + secret: ${{ github.TOKEN }} + approvers: AlexZorkin,emi-hi,tim738745,kuanfandevops,jig-patel,prv-proton,JulianForeman + minimum-approvals: 1 + issue-title: "TFRS main-release-jan-2024 Test Deployment" + + deploy-on-test: + name: Deploy on Test + needs: approval-test-deployment + uses: ./.github/workflows/branch-deploy-template.yaml + with: + branch-name: main-release-jan-2024 + suffix: -test-jan + env-name: test + database-service-host-name: tfrs-crunchy-test-pgbouncer + rabbitmq-vhost: tfrs-jan-vhost + secrets: + tools-namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-tools + namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-test + openshift-server: ${{ secrets.OPENSHIFT_SERVER }} + openshift-token: ${{ secrets.OPENSHIFT_TOKEN }} \ No newline at end of file diff --git a/.github/workflows/pr-dev-cicd.yaml b/.github/workflows/pr-dev-cicd.yaml new file mode 100644 index 000000000..ce6128835 --- /dev/null +++ b/.github/workflows/pr-dev-cicd.yaml @@ -0,0 +1,54 @@ +# Please refer to ./readme.md for how to build single pull request + +# Update this workflow name per pull request +name: TFRS Dev Jan PR CICD +on: + workflow_dispatch: + pull_request: + types: [opened, edited, synchronize, reopened] + branches: + - 'main-release-jan-2024' + +jobs: + + setup-database: + if: endsWith( github.event.pull_request.title, 'build-on-dev' ) + uses: ./.github/workflows/pr-dev-database-template.yaml + with: + pr-number: ${{ github.event.pull_request.number }} + dev-suffix: -jan-${{ github.event.pull_request.number }} + secrets: + dev-namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-dev + tfrs-dev-username: ${{ secrets.TFRS_DEV_USERNAME }} + tfrs-dev-password: ${{ secrets.TFRS_DEV_PASSWORD }} + openshift-server: ${{ secrets.OPENSHIFT_SERVER }} + openshift-token: ${{ secrets.OPENSHIFT_TOKEN }} + + # when build pull reuqest, the suffix sample is -jan-1234 + # the checkout-ref is in the format of refs/pull/1234/head + build: + if: endsWith( github.event.pull_request.title, 'build-on-dev' ) + name: Build Pull Request + uses: ./.github/workflows/build-template.yaml + with: + suffix: -jan-${{ github.event.pull_request.number }} + checkout-ref: refs/pull/${{ github.event.pull_request.number }}/head + secrets: + tools-namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-tools + openshift-server: ${{ secrets.OPENSHIFT_SERVER }} + openshift-token: ${{ secrets.OPENSHIFT_TOKEN }} + + deploy: + if: endsWith( github.event.pull_request.title, 'build-on-dev' ) + needs: [setup-database, build] + uses: ./.github/workflows/pr-dev-deploy-template.yaml + with: + suffix: -jan-${{ github.event.pull_request.number }} + checkout-ref: refs/pull/${{ github.event.pull_request.number }}/head + database-service-host-name: tfrs-spilo-jan-${{ github.event.pull_request.number }} + secrets: + tools-namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-tools + namespace: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-dev + openshift-server: ${{ secrets.OPENSHIFT_SERVER }} + openshift-token: ${{ secrets.OPENSHIFT_TOKEN }} + \ No newline at end of file diff --git a/.github/workflows/pr-dev-database-template.yaml b/.github/workflows/pr-dev-database-template.yaml new file mode 100644 index 000000000..458d8b62b --- /dev/null +++ b/.github/workflows/pr-dev-database-template.yaml @@ -0,0 +1,69 @@ +name: PR Dev Database Template + +on: + workflow_call: + inputs: + # pull request number + pr-number: + required: true + type: string + # the suffix will be appended to tfrs-spilo, same values: -1234, -jan-1242 + dev-suffix: + required: true + type: string + secrets: + dev-namespace: + required: true + tfrs-dev-username: + required: true + tfrs-dev-password: + required: true + openshift-server: + required: true + openshift-token: + required: true + +jobs: + + database: + + name: Start Database + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3 + with: + ref: refs/pull/${{ inputs.pr-number }}/head + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.dev-namespace }} + + - name: Setup Database + shell: bash {0} + run: | + cd charts/tfrs-spilo + helm dependency build + helm status -n ${{ secrets.dev-namespace }} tfrs-spilo${{ inputs.dev-suffix }} + if [ $? -eq 0 ]; then + echo "tfrs-spilo${{ inputs.dev-suffix }} exists already" + else + echo "Installing tfrs-spilo${{ inputs.dev-suffix }}" + helm install -n ${{ secrets.dev-namespace }} -f ./values-dev.yaml --wait tfrs-spilo${{ inputs.dev-suffix }} . + oc -n ${{ secrets.dev-namespace }} wait --for=condition=Ready pod/tfrs-spilo${{ inputs.dev-suffix }}-0 + oc -n ${{ secrets.dev-namespace }} exec tfrs-spilo${{ inputs.dev-suffix }}-0 -- psql -c "create user \"${{ secrets.tfrs-dev-username }}\" WITH PASSWORD '${{ secrets.tfrs-dev-password }}'" || true + oc -n ${{ secrets.dev-namespace }} exec tfrs-spilo${{ inputs.dev-suffix }}-0 -- psql -c "create database tfrs owner \"${{ secrets.tfrs-dev-username }}\" ENCODING 'UTF8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8'" || true + oc -n ${{ secrets.dev-namespace }} exec tfrs-spilo${{ inputs.dev-suffix }}-0 -- psql -c "ALTER SYSTEM SET log_filename='postgresql-%H.log'" || true + oc -n ${{ secrets.dev-namespace }} exec tfrs-spilo${{ inputs.dev-suffix }}-0 -- psql -c "ALTER SYSTEM SET log_connections='off'" || true + oc -n ${{ secrets.dev-namespace }} exec tfrs-spilo${{ inputs.dev-suffix }}-0 -- psql -c "ALTER SYSTEM SET log_disconnections='off'" || true + oc -n ${{ secrets.dev-namespace }} exec tfrs-spilo${{ inputs.dev-suffix }}-0 -- psql -c "ALTER SYSTEM SET log_checkpoints='off'" || true + oc -n ${{ secrets.dev-namespace }} exec tfrs-spilo${{ inputs.dev-suffix }}-0 -- psql -c "select pg_reload_conf()" || true + fi + diff --git a/.github/workflows/pr-dev-deploy-template.yaml b/.github/workflows/pr-dev-deploy-template.yaml new file mode 100644 index 000000000..ddc583c30 --- /dev/null +++ b/.github/workflows/pr-dev-deploy-template.yaml @@ -0,0 +1,191 @@ + + +name: PR Dev Deploy Template + +on: + workflow_call: + inputs: + # suffix is in format of -jan-1923 + suffix: + required: true + type: string + # when build pull request, the sample value is refs/pull/2023/head + checkout-ref: + required: true + type: string + # database-service-host-name, sample tfrs-spilo-dev-1988 + database-service-host-name: + required: true + type: string + secrets: + tools-namespace: + required: true + namespace: + required: true + openshift-server: + required: true + openshift-token: + required: true + +jobs: + + deploy: + + name: Deploy tfrs + runs-on: ubuntu-latest + timeout-minutes: 60 + + steps: + + - name: Check out repository + uses: actions/checkout@v3 + with: + ref: ${{ inputs.checkout-ref }} + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.openshift-server }} + openshift_token: ${{ secrets.openshift-token }} + insecure_skip_tls_verify: true + namespace: ${{ secrets.tools-namespace }} + + - name: Create vhost on Rabbitmq Dev + shell: bash {0} + run: | + oc -n ${{ secrets.namespace }} exec tfrs-rabbitmq-0 -- rabbitmqctl add_vhost tfrs-dev${{ inputs.suffix }}-vhost + oc -n ${{ secrets.namespace }} exec tfrs-rabbitmq-0 -- rabbitmqctl set_permissions --vhost tfrs-dev${{ inputs.suffix }}-vhost tfrs ".*" ".*" ".*" + + - name: Deploy tfrs-frontend + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-frontend:build${{ inputs.suffix }} ${{ secrets.namespace }}/tfrs-frontend:dev${{ inputs.suffix }} + cd charts/tfrs-apps/charts/tfrs-frontend + helm status -n ${{ secrets.namespace }} tfrs-frontend-dev${{ inputs.suffix }} + if [ $? -eq 0 ]; then + echo "tfrs-frontend-dev${{ inputs.suffix }} release exists already" + helm upgrade \ + --set frontendImageTagName=dev${{ inputs.suffix }} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-frontend-dev${{ inputs.suffix }} . + else + echo "tfrs-frontend-dev${{ inputs.suffix }} release does not exist" + helm install \ + --set frontendImageTagName=dev${{ inputs.suffix }} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-frontend-dev${{ inputs.suffix }} . + fi + + - name: Deploy tfrs-backend + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-backend:build${{ inputs.suffix }} ${{ secrets.namespace }}/tfrs-backend:dev${{ inputs.suffix }} + cd charts/tfrs-apps/charts/tfrs-backend + helm status -n ${{ secrets.namespace }} tfrs-backend-dev${{ inputs.suffix }} + if [ $? -eq 0 ]; then + echo "tfrs-backend-dev${{ inputs.suffix }} release exists already" + helm upgrade \ + --set backendImageTagName=dev${{ inputs.suffix }} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=tfrs-dev${{ inputs.suffix }}-vhost \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-backend-dev${{ inputs.suffix }} . + else + echo "tfrs-backend-dev${{ inputs.suffix }} release does not exist" + helm install \ + --set backendImageTagName=dev${{ inputs.suffix }} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=tfrs-dev${{ inputs.suffix }}-vhost \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-backend-dev${{ inputs.suffix }} . + fi + + - name: Deploy tfrs-celery + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-celery:build${{ inputs.suffix }} ${{ secrets.namespace }}/tfrs-celery:dev${{ inputs.suffix }} + cd charts/tfrs-apps/charts/tfrs-celery + helm status -n ${{ secrets.namespace }} tfrs-celery-dev${{ inputs.suffix }} + if [ $? -eq 0 ]; then + echo "tfrs-celery-dev${{ inputs.suffix }} release exists already" + helm upgrade \ + --set celeryImageTagName=dev${{ inputs.suffix }} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=tfrs-dev${{ inputs.suffix }}-vhost \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-celery-dev${{ inputs.suffix }} . + else + echo "tfrs-celery-dev${{ inputs.suffix }} release does not exist" + helm install \ + --set celeryImageTagName=dev${{ inputs.suffix }} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=tfrs-dev${{ inputs.suffix }}-vhost \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-celery-dev${{ inputs.suffix }} . + fi + + - name: Deploy tfrs-scan-handler + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-scan-handler:build${{ inputs.suffix }} ${{ secrets.namespace }}/tfrs-scan-handler:dev${{ inputs.suffix }} + cd charts/tfrs-apps/charts/tfrs-scan-handler + helm status -n ${{ secrets.namespace }} tfrs-scan-handler-dev${{ inputs.suffix }} + if [ $? -eq 0 ]; then + echo "tfrs-scan-handler-dev${{ inputs.suffix }} release exists already" + helm upgrade \ + --set scanHandlerImageTagName=dev${{ inputs.suffix }} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=tfrs-dev${{ inputs.suffix }}-vhost \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-scan-handler-dev${{ inputs.suffix }} . + else + echo "tfrs-scan-handler-dev${{ inputs.suffix }} release does not exist" + helm install \ + --set scanHandlerImageTagName=dev${{ inputs.suffix }} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + --set databaseServiceHostName=${{ inputs.database-service-host-name }} \ + --set rabbitmqVHost=tfrs-dev${{ inputs.suffix }}-vhost \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-scan-handler-dev${{ inputs.suffix }} . + fi + + - name: Deploy tfrs-scan-coordinator + shell: bash {0} + run: | + oc tag ${{ secrets.tools-namespace }}/tfrs-scan-coordinator:build${{ inputs.suffix}} ${{ secrets.namespace }}/tfrs-scan-coordinator:dev${{ inputs.suffix}} + cd charts/tfrs-apps/charts/tfrs-scan-coordinator + helm status -n ${{ secrets.namespace }} tfrs-scan-coordinator-dev${{ inputs.suffix }} + if [ $? -eq 0 ]; then + echo "tfrs-scan-coordinator-dev${{ inputs.suffix }} release exists already" + helm upgrade \ + --set scanCoordinatorImageTagName=dev${{ inputs.suffix}} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + --set rabbitmqVHost=tfrs-dev${{ inputs.suffix }}-vhost \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-scan-coordinator-dev${{ inputs.suffix }} . + else + echo "tfrs-scan-coordinator${{ inputs.suffix }} release does not exist" + helm install \ + --set scanCoordinatorImageTagName=dev${{ inputs.suffix}} \ + --set suffix=-dev${{ inputs.suffix }} \ + --set namespace=${{ secrets.namespace }} \ + --set envName=dev \ + --set rabbitmqVHost=tfrs-dev${{ inputs.suffix }}-vhost \ + -n ${{ secrets.namespace }} -f ./values-dev-jan.yaml tfrs-scan-coordinator-dev${{ inputs.suffix }} . + fi \ No newline at end of file diff --git a/.github/workflows/pr-teardown.yaml b/.github/workflows/pr-teardown.yaml new file mode 100644 index 000000000..f6e426cbd --- /dev/null +++ b/.github/workflows/pr-teardown.yaml @@ -0,0 +1,41 @@ +name: TFRS Dev Jan PR Teardown + +on: + pull_request: + types: closed + branches: + - 'main-release-jan-2024' + +env: + TOOLS_NAMESPACE: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-tools + DEV_NAMESPACE: ${{ secrets.OPENSHIFT_NAMESPACE_PLATE }}-dev + +jobs: + + teardown-on-dev: + if: endsWith( github.event.pull_request.title, 'build-on-dev' ) + name: Tear TFRS down on Dev + runs-on: ubuntu-latest + timeout-minutes: 20 + + steps: + + - name: Log in to Openshift + uses: redhat-actions/oc-login@v1.2 + with: + openshift_server_url: ${{ secrets.OPENSHIFT_SERVER }} + openshift_token: ${{ secrets.OPENSHIFT_TOKEN }} + insecure_skip_tls_verify: true + namespace: ${{ env.TOOLS_NAMESPACE }} + + - name: Undeploy on Dev + shell: bash {0} + run: | + oc -n ${{ env.DEV_NAMESPACE }} exec tfrs-rabbitmq-0 -- rabbitmqctl delete_vhost tfrs-dev-jan-${{ github.event.pull_request.number }}-vhost + helm -n ${{ env.DEV_NAMESPACE }} uninstall tfrs-spilo-jan-${{ github.event.pull_request.number }} || true + helm -n ${{ env.DEV_NAMESPACE }} uninstall tfrs-backend-dev-jan-${{ github.event.pull_request.number }} || true + helm -n ${{ env.DEV_NAMESPACE }} uninstall tfrs-frontend-dev-jan-${{ github.event.pull_request.number }} || true + helm -n ${{ env.DEV_NAMESPACE }} uninstall tfrs-celery-dev-jan-${{ github.event.pull_request.number }} || true + helm -n ${{ env.DEV_NAMESPACE }} uninstall tfrs-scan-handler-dev-jan-${{ github.event.pull_request.number }} || true + helm -n ${{ env.DEV_NAMESPACE }} uninstall tfrs-scan-coordinator-dev-jan-${{ github.event.pull_request.number }} || true + diff --git a/README.md b/README.md index e05c7acc3..c6a689b58 100644 --- a/README.md +++ b/README.md @@ -120,3 +120,4 @@ This is a list that was created on 2023-02-01 with all Zelda Devs to provide alt - New learning and applying it to our work - Innovation work + diff --git a/backend/api/fixtures/test/test_carbon_intensity_limits.json b/backend/api/fixtures/test/test_carbon_intensity_limits.json index 44bdd22f5..beec13ceb 100644 --- a/backend/api/fixtures/test/test_carbon_intensity_limits.json +++ b/backend/api/fixtures/test/test_carbon_intensity_limits.json @@ -70,4 +70,41 @@ }, "model": "api.carbonintensitylimit", "pk": null -}] \ No newline at end of file +}, { + "fields": { + "compliance_period": ["2023"], + "fuel_class": ["Diesel"], + "density": "79.64", + "effective_date": "2023-01-01" + }, + "model": "api.carbonintensitylimit", + "pk": null +}, { + "fields": { + "compliance_period": ["2023"], + "fuel_class": ["Gasoline"], + "density": "74.08", + "effective_date": "2023-01-01" + }, + "model": "api.carbonintensitylimit", + "pk": null +}, { + "fields": { + "compliance_period": ["2022"], + "fuel_class": ["Diesel"], + "density": "79.64", + "effective_date": "2022-01-01" + }, + "model": "api.carbonintensitylimit", + "pk": null +}, { + "fields": { + "compliance_period": ["2022"], + "fuel_class": ["Gasoline"], + "density": "74.08", + "effective_date": "2022-01-01" + }, + "model": "api.carbonintensitylimit", + "pk": null +} +] \ No newline at end of file diff --git a/backend/api/fixtures/test/test_fuel_codes.json b/backend/api/fixtures/test/test_fuel_codes.json index 9f54f969f..54263dabd 100644 --- a/backend/api/fixtures/test/test_fuel_codes.json +++ b/backend/api/fixtures/test/test_fuel_codes.json @@ -20,6 +20,30 @@ }, "model": "api.fuelcode", "pk": 1 +}, +{ + "model": "api.fuelcode", + "pk": 21, + "fields": { + "fuel_code": "BCLCF", + "fuel_code_version": "114", + "fuel_code_version_minor": "0", + "company": "ETH Alco", + "carbon_intensity": "38.12", + "application_date": "2021-09-01", + "effective_date": "2021-09-02", + "expiry_date": "2024-09-01", + "fuel": ["LNG"], + "feedstock": "MSW", + "feedstock_location": "Test", + "feedstock_misc": "", + "facility_location": "Test", + "facility_nameplate": "654", + "former_company": "", + "approval_date": "2021-09-02", + "status": ["Approved"], + "renewable_percentage": "50.00" + } }, { "fields": { diff --git a/backend/api/fixtures/test/test_post_compliance_unit_reporting.json b/backend/api/fixtures/test/test_post_compliance_unit_reporting.json new file mode 100644 index 000000000..eacbd745b --- /dev/null +++ b/backend/api/fixtures/test/test_post_compliance_unit_reporting.json @@ -0,0 +1,378 @@ +[ + { + "fields": { + "fuel_supplier_status": "Draft", + "analyst_status": "Unreviewed", + "director_status": "Unreviewed", + "manager_status": "Unreviewed" + }, + "model": "api.compliancereportworkflowstate", + "pk": 1 + }, + { + "fields": { + "fuel_supplier_status": "Submitted", + "analyst_status": "Unreviewed", + "director_status": "Unreviewed", + "manager_status": "Unreviewed" + }, + "model": "api.compliancereportworkflowstate", + "pk": 2 + }, + { + "fields": { + "fuel_supplier_status": "Draft", + "analyst_status": "Unreviewed", + "director_status": "Unreviewed", + "manager_status": "Unreviewed" + }, + "model": "api.compliancereportworkflowstate", + "pk": 3 + }, + { + "fields": { + "status": 1, + "type": [ + "Compliance Report" + ], + "organization": [ + "Test Org 1" + ], + "compliance_period": [ + "2018" + ], + "create_timestamp": "2019-01-01", + "update_timestamp": "2019-01-01", + "root_report": 1, + "latest_report": 1, + "traversal": 0 + }, + "model": "api.compliancereport", + "pk": 1 + }, + { + "fields": { + "status": 2, + "type": [ + "Exclusion Report" + ], + "organization": [ + "Test Org 1" + ], + "compliance_period": [ + "2018" + ], + "create_timestamp": "2019-01-01", + "update_timestamp": "2019-01-01", + "root_report": 2, + "latest_report": 2, + "traversal": 0 + }, + "model": "api.compliancereport", + "pk": 2 + }, + { + "fields": { + "status": 3, + "type": [ + "Compliance Report" + ], + "organization": [ + "Test Org 1" + ], + "compliance_period": [ + "2019" + ], + "create_timestamp": "2019-01-01", + "update_timestamp": "2019-01-01", + "root_report": 3, + "latest_report": 3, + "traversal": 0 + }, + "model": "api.compliancereport", + "pk": 3 + }, + { + "fields": { + "role": [ + "ComplianceReporting" + ], + "user": [ + "fs_user_1" + ] + }, + "model": "api.userrole", + "pk": null + }, + { + "fields": { + "description": "Other", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.expecteduse", + "pk": 3 + }, + { + "fields": { + "the_type": "Received", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.notionaltransfertype", + "pk": 1 + }, + { + "fields": { + "the_type": "Carbon Intensity", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 1 + }, + { + "fields": { + "the_type": "Fuel Code", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 2 + }, + { + "fields": { + "the_type": "Default Carbon Intensity", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 3 + }, + { + "fields": { + "the_type": "GHGenius", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 4 + }, + { + "fields": { + "the_type": "Alternative", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 5 + }, + { + "model": "api.provisionoftheact", + "pk": 5, + "fields": { + "display_order": 5, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (a)", + "description": "Prescribed carbon intensity" + } + }, + { + "model": "api.provisionoftheact", + "pk": 6, + "fields": { + "display_order": 6, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (b)", + "description": "Prescribed carbon intensity" + } + }, + { + "model": "api.provisionoftheact", + "pk": 2, + "fields": { + "display_order": 99, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (c)", + "description": "Approved fuel code" + } + }, + { + "model": "api.provisionoftheact", + "pk": 4, + "fields": { + "display_order": 4, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (d) (i)", + "description": "Default Carbon Intensity Value" + } + }, + { + "model": "api.provisionoftheact", + "pk": 3, + "fields": { + "display_order": 99, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (d) (ii) (A)", + "description": "GHGenius modelled" + } + }, + { + "model": "api.provisionoftheact", + "pk": 1, + "fields": { + "display_order": 99, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (d) (ii) (B)", + "description": "Alternative Method" + } + }, + { + "model": "api.approvedfuelprovision", + "pk": 11, + "fields": { + "fuel": ["LNG"], + "provision_act": 2, + "determination_type": ["Carbon Intensity"] + } + }, + { + "model": "api.approvedfuelprovision", + "pk": 12, + "fields": { + "fuel": ["CNG"], + "provision_act": 2, + "determination_type": ["Fuel Code"] + } + }, + { + "model": "api.approvedfuelprovision", + "pk": 13, + "fields": { + "fuel": ["LNG"], + "provision_act": 6, + "determination_type": ["Fuel Code"] + } + }, + { + "model": "api.approvedfuelprovision", + "pk": 14, + "fields": { + "fuel": ["CNG"], + "provision_act": 6, + "determination_type": ["Carbon Intensity"] + } + }, + { + "model": "api.energyeffectivenessratiocategory", + "pk": 6, + "fields": { + "display_order": 6, + "name": "Petroleum-based gasoline, natural gas-based gasoline or renewable fuel in relation to gasoline class fuel" + } + }, + { + "model": "api.energyeffectivenessratiocategory", + "pk": 5, + "fields": { + "display_order": 5, + "name": "'Petroleum-based diesel fuel or renewable fuel in relation to diesel class fuel'" + } + }, + { + "model": "api.energyeffectivenessratio", + "pk": 1, + "fields": { + "effective_date": "2016-01-01", + "expiration_date": "2023-12-31", + "category": 6, + "fuel_class": ["Gasoline"], + "ratio": "1.00" + } + }, + { + "model": "api.energyeffectivenessratio", + "pk": 2, + "fields": { + "effective_date": "2016-01-01", + "expiration_date": "2023-12-31", + "category": 5, + "fuel_class": ["Diesel"], + "ratio": "1.00" + } + }, + { + "model": "api.energydensity", + "pk": 4, + "fields": { + "effective_date": "2017-01-01", + "category": 4, + "density": "23.58" + } + }, + { + "model": "api.energydensity", + "pk": 9, + "fields": { + "effective_date": "2017-01-01", + "category": 9, + "density": "38.65" + } + }, + { + "model": "api.approvedfuel", + "pk": 4, + "fields": { + "effective_date": "2017-01-01", + "name": "Ethanol", + "description": "Ethanol produced from biomass", + "credit_calculation_only": false, + "default_carbon_intensity_category": 10, + "energy_density_category": 4, + "energy_effectiveness_ratio_category": 6, + "unit_of_measure": ["L"], + "is_partially_renewable": true + } + }, + { + "model": "api.approvedfuel", + "pk": 19, + "fields": { + "effective_date": "2017-01-01", + "name": "'Petroleum-based diesel'", + "description": "'Diesel fuel, diesel, petroleum-based diesel'", + "credit_calculation_only": true, + "default_carbon_intensity_category": 6, + "energy_density_category": 9, + "energy_effectiveness_ratio_category": 5, + "unit_of_measure": ["L"], + "is_partially_renewable": false + } + }, + { + "fields": { + "determination_type": ["Carbon Intensity"], + "provision_act": 1, + "fuel": ["CNG"] + }, + "model": "api.approvedfuelprovision", + "pk": 500 + }, + { + "fields": { + "determination_type": ["GHGenius"], + "provision_act": 3, + "fuel": ["LNG"] + }, + "model": "api.approvedfuelprovision" + }, + { + "fields": { + "determination_type": ["Carbon Intensity"], + "provision_act": 1, + "fuel": ["LNG"] + }, + "model": "api.approvedfuelprovision", + "pk": 500 + } +] diff --git a/backend/api/fixtures/test/test_pre_compliance_unit_reporting.json b/backend/api/fixtures/test/test_pre_compliance_unit_reporting.json new file mode 100644 index 000000000..4a98f0045 --- /dev/null +++ b/backend/api/fixtures/test/test_pre_compliance_unit_reporting.json @@ -0,0 +1,378 @@ +[ + { + "fields": { + "fuel_supplier_status": "Draft", + "analyst_status": "Unreviewed", + "director_status": "Unreviewed", + "manager_status": "Unreviewed" + }, + "model": "api.compliancereportworkflowstate", + "pk": 1 + }, + { + "fields": { + "fuel_supplier_status": "Submitted", + "analyst_status": "Unreviewed", + "director_status": "Unreviewed", + "manager_status": "Unreviewed" + }, + "model": "api.compliancereportworkflowstate", + "pk": 2 + }, + { + "fields": { + "fuel_supplier_status": "Draft", + "analyst_status": "Unreviewed", + "director_status": "Unreviewed", + "manager_status": "Unreviewed" + }, + "model": "api.compliancereportworkflowstate", + "pk": 3 + }, + { + "fields": { + "status": 1, + "type": [ + "Compliance Report" + ], + "organization": [ + "Test Org 1" + ], + "compliance_period": [ + "2018" + ], + "create_timestamp": "2019-01-01", + "update_timestamp": "2019-01-01", + "root_report": 1, + "latest_report": 1, + "traversal": 0 + }, + "model": "api.compliancereport", + "pk": 1 + }, + { + "fields": { + "status": 2, + "type": [ + "Exclusion Report" + ], + "organization": [ + "Test Org 1" + ], + "compliance_period": [ + "2018" + ], + "create_timestamp": "2019-01-01", + "update_timestamp": "2019-01-01", + "root_report": 2, + "latest_report": 2, + "traversal": 0 + }, + "model": "api.compliancereport", + "pk": 2 + }, + { + "fields": { + "status": 3, + "type": [ + "Compliance Report" + ], + "organization": [ + "Test Org 1" + ], + "compliance_period": [ + "2019" + ], + "create_timestamp": "2019-01-01", + "update_timestamp": "2019-01-01", + "root_report": 3, + "latest_report": 3, + "traversal": 0 + }, + "model": "api.compliancereport", + "pk": 3 + }, + { + "fields": { + "role": [ + "ComplianceReporting" + ], + "user": [ + "fs_user_1" + ] + }, + "model": "api.userrole", + "pk": null + }, + { + "fields": { + "description": "Other", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.expecteduse", + "pk": 3 + }, + { + "fields": { + "the_type": "Received", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.notionaltransfertype", + "pk": 1 + }, + { + "fields": { + "the_type": "Carbon Intensity", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 1 + }, + { + "fields": { + "the_type": "Fuel Code", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 2 + }, + { + "fields": { + "the_type": "Default Carbon Intensity", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 3 + }, + { + "fields": { + "the_type": "GHGenius", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 4 + }, + { + "fields": { + "the_type": "Alternative", + "display_order": 99, + "effective_date": "2017-01-01" + }, + "model": "api.carbonintensitydeterminationtype", + "pk": 5 + }, + { + "model": "api.provisionoftheact", + "pk": 5, + "fields": { + "display_order": 5, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (a)", + "description": "Prescribed carbon intensity" + } + }, + { + "model": "api.provisionoftheact", + "pk": 6, + "fields": { + "display_order": 6, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (b)", + "description": "Prescribed carbon intensity" + } + }, + { + "model": "api.provisionoftheact", + "pk": 2, + "fields": { + "display_order": 99, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (c)", + "description": "Approved fuel code" + } + }, + { + "model": "api.provisionoftheact", + "pk": 4, + "fields": { + "display_order": 4, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (d) (i)", + "description": "Default Carbon Intensity Value" + } + }, + { + "model": "api.provisionoftheact", + "pk": 3, + "fields": { + "display_order": 99, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (d) (ii) (A)", + "description": "GHGenius modelled" + } + }, + { + "model": "api.provisionoftheact", + "pk": 1, + "fields": { + "display_order": 99, + "effective_date": "2017-01-01", + "provision": "Section 6 (5) (d) (ii) (B)", + "description": "Alternative Method" + } + }, + { + "model": "api.approvedfuelprovision", + "pk": 11, + "fields": { + "fuel": ["LNG"], + "provision_act": 2, + "determination_type": ["Carbon Intensity"] + } + }, + { + "model": "api.approvedfuelprovision", + "pk": 12, + "fields": { + "fuel": ["CNG"], + "provision_act": 2, + "determination_type": ["Fuel Code"] + } + }, + { + "model": "api.approvedfuelprovision", + "pk": 13, + "fields": { + "fuel": ["LNG"], + "provision_act": 6, + "determination_type": ["Fuel Code"] + } + }, + { + "model": "api.approvedfuelprovision", + "pk": 14, + "fields": { + "fuel": ["CNG"], + "provision_act": 6, + "determination_type": ["Carbon Intensity"] + } + }, + { + "model": "api.energyeffectivenessratiocategory", + "pk": 6, + "fields": { + "display_order": 6, + "name": "Petroleum-based gasoline, natural gas-based gasoline or renewable fuel in relation to gasoline class fuel" + } + }, + { + "model": "api.energyeffectivenessratiocategory", + "pk": 5, + "fields": { + "display_order": 5, + "name": "'Petroleum-based diesel fuel or renewable fuel in relation to diesel class fuel'" + } + }, + { + "model": "api.energyeffectivenessratio", + "pk": 1, + "fields": { + "effective_date": "2016-01-01", + "expiration_date": "2023-12-31", + "category": 6, + "fuel_class": ["Gasoline"], + "ratio": "1.00" + } + }, + { + "model": "api.energyeffectivenessratio", + "pk": 2, + "fields": { + "effective_date": "2016-01-01", + "expiration_date": "2023-12-31", + "category": 5, + "fuel_class": ["Diesel"], + "ratio": "1.00" + } + }, + { + "model": "api.energydensity", + "pk": 4, + "fields": { + "effective_date": "2017-01-01", + "category": 4, + "density": "23.58" + } + }, + { + "model": "api.energydensity", + "pk": 9, + "fields": { + "effective_date": "2017-01-01", + "category": 9, + "density": "38.65" + } + }, + { + "model": "api.approvedfuel", + "pk": 4, + "fields": { + "effective_date": "2017-01-01", + "name": "Ethanol", + "description": "Ethanol produced from biomass", + "credit_calculation_only": false, + "default_carbon_intensity_category": 10, + "energy_density_category": 4, + "energy_effectiveness_ratio_category": 6, + "unit_of_measure": ["L"], + "is_partially_renewable": true + } + }, + { + "model": "api.approvedfuel", + "pk": 19, + "fields": { + "effective_date": "2017-01-01", + "name": "'Petroleum-based diesel'", + "description": "'Diesel fuel, diesel, petroleum-based diesel'", + "credit_calculation_only": true, + "default_carbon_intensity_category": 6, + "energy_density_category": 9, + "energy_effectiveness_ratio_category": 5, + "unit_of_measure": ["L"], + "is_partially_renewable": false + } + }, + { + "fields": { + "determination_type": ["Carbon Intensity"], + "provision_act": 1, + "fuel": ["CNG"] + }, + "model": "api.approvedfuelprovision", + "pk": 500 + }, + { + "fields": { + "determination_type": ["GHGenius"], + "provision_act": 3, + "fuel": ["LNG"] + }, + "model": "api.approvedfuelprovision" + }, + { + "fields": { + "determination_type": ["Carbon Intensity"], + "provision_act": 1, + "fuel": ["LNG"] + }, + "model": "api.approvedfuelprovision", + "pk": 500 + } +] diff --git a/backend/api/migrations/0014_new_label_changes.py b/backend/api/migrations/0014_new_label_changes.py new file mode 100644 index 000000000..5f536ccd7 --- /dev/null +++ b/backend/api/migrations/0014_new_label_changes.py @@ -0,0 +1,216 @@ +from django.db import migrations +from django.db.migrations import RunPython + + +def update_permissions(apps, schema_editor): + """ + Updates the permissions and removes the create/edit fuel suppliers + from Government Analyst and Director roles + """ + db_alias = schema_editor.connection.alias + + permission = apps.get_model("api", "Permission") + notification_subscription = apps.get_model("api", "NotificationSubscription") + permission.objects.using(db_alias).filter( + code="VIEW_CREDIT_TRANSFERS" + ).update( + name="View compliance unit transactions", + description="View compliance unit transactions" + ) + + permission.objects.using(db_alias).filter( + code="VIEW_APPROVED_CREDIT_TRANSFERS" + ).update( + name="View recorded compliance unit transactions", + description="view compliance unit transactions within the Historical " + "Data Entry tool prior to them being committed" + ) + + permission.objects.using(db_alias).filter( + code="CREDIT_CALCULATION_MANAGE" + ).update( + name="Edit compliance unit calculation values", + description="edit values used in compliance unit calculation formula" + ) + + permission.objects.using(db_alias).filter( + code="DOCUMENTS_LINK_TO_CREDIT_TRADE" + ).update( + name="Link file submissions to compliance unit transactions", + description="establish link between file submissions and compliance unit transactions" + ) + + permission.objects.using(db_alias).filter( + code="RECOMMEND_CREDIT_TRANSFER" + ).update( + name="Recommend Credit transfers and Initiative Agreement submissions", + description="Make recommendations to the director for Credit transfers and Initiative Agreement submissions" + ) + + permission.objects.using(db_alias).filter( + code="APPROVE_CREDIT_TRANSFER" + ).update( + name="Record credit transfers and issue credits under Initiative Agreements", + description="Record credit transfers and issue credits under Initiative Agreements" + ) + + permission.objects.using(db_alias).filter( + code="DECLINE_CREDIT_TRANSFER" + ).update( + name="Decline credit transfers Initiative Agreement submissions", + description="Decline to record credit transfers and decline to issue credits under Initiative Agreements" + ) + + permission.objects.using(db_alias).filter( + code="PROPOSE_CREDIT_TRANSFER" + ).update( + name="Create new credit transfer", + description="Create new credit transfer" + ) + + permission.objects.using(db_alias).filter( + code="RESCIND_CREDIT_TRANSFER" + ).update( + name="Rescind a credit transfer", + description="Rescind a credit transfer sent to another organization" + ) + + permission.objects.using(db_alias).filter( + code="CREDIT_CALCULATION_VIEW" + ).update( + name="View compliance unit calculation values", + description="View values used in compliance unit calculation formula" + ) + + permission.objects.using(db_alias).filter( + code="REFUSE_CREDIT_TRANSFER" + ).update( + name="Refuse a credit transfer", + description="Refuse a credit transfer proposed by another organization" + ) + + permission.objects.using(db_alias).filter( + code="SIGN_CREDIT_TRANSFER" + ).update( + name="Propose and accept credit transfers", + description="Propose and accept credit transfers" + ) + + permission.objects.using(db_alias).filter( + code="USE_HISTORICAL_DATA_ENTRY" + ).update( + name="Use Historical Data Entry", + description="Record compliance unit transactions approved outside of TFRS" + ) + +def revert_permissions(apps, schema_editor): + """ + Reverts the permission back to its previous state by assigning + back the permission back to Government Analyst and Director + """ + db_alias = schema_editor.connection.alias + + permission = apps.get_model("api", "Permission") + permission.objects.using(db_alias).filter( + code="VIEW_CREDIT_TRANSFERS" + ).update( + name="View credit transactions", + description="View credit transactions" + ) + + permission.objects.using(db_alias).filter( + code="VIEW_APPROVED_CREDIT_TRANSFERS" + ).update( + name="View recorded credit transactions", + description="view credit transactions within the Historical " + "Data Entry tool prior to them being committed" + ) + + permission.objects.using(db_alias).filter( + code="CREDIT_CALCULATION_MANAGE" + ).update( + name="Edit credit calculation values", + description="edit values used in credit calculation formula" + ) + + permission.objects.using(db_alias).filter( + code="DOCUMENTS_LINK_TO_CREDIT_TRADE" + ).update( + name="Link file submissions to Part 3 Awards", + description="establish link between file submissions and Part 3 Awards" + ) + + permission.objects.using(db_alias).filter( + code="RECOMMEND_CREDIT_TRANSFER" + ).update( + name="Recommend Credit Transfer Proposals and Part 3 Awards", + description="recommend or not recommend approval of Credit Transfer Proposals and Part 3 Awards" + ) + + permission.objects.using(db_alias).filter( + code="APPROVE_CREDIT_TRANSFER" + ).update( + name="Approve credit transfer proposals and Part 3 Awards", + description="approve credit transfer proposals and Part 3 Awards" + ) + + permission.objects.using(db_alias).filter( + code="DECLINE_CREDIT_TRANSFER" + ).update( + name="Decline to approve credit transfer proposals and Part 3 Awards", + description="decline credit transfer proposals and Part 3 Awards" + ) + + permission.objects.using(db_alias).filter( + code="PROPOSE_CREDIT_TRANSFER" + ).update( + name="Create New Credit Transfer Proposal", + description="create new credit transfer proposal" + ) + + permission.objects.using(db_alias).filter( + code="RESCIND_CREDIT_TRANSFER" + ).update( + name="Rescind a credit transfer proposal", + description="rescind a credit transfer proposal sent to another organization" + ) + + permission.objects.using(db_alias).filter( + code="CREDIT_CALCULATION_VIEW" + ).update( + name="View credit calculation values", + description="view values used in credit calculation formula" + ) + + permission.objects.using(db_alias).filter( + code="REFUSE_CREDIT_TRANSFER" + ).update( + name="Refuse a credit transfer proposal", + description="refuse a credit transfer proposal received from another organization" + ) + + permission.objects.using(db_alias).filter( + code="SIGN_CREDIT_TRANSFER" + ).update( + name="Propose and accept credit transfer proposals", + description="propose and accept credit transfer proposals" + ) + + permission.objects.using(db_alias).filter( + code="USE_HISTORICAL_DATA_ENTRY" + ).update( + name="Use Historical Data Entry", + description="Record credit transactions approved outside of TFRS" + ) + +class Migration(migrations.Migration): + """ + Attaches the functions for the migrations + """ + dependencies = [ + ('api', '0013_create_missing_rescinded_history_records'), + ] + + operations = [ + RunPython(update_permissions, revert_permissions) + ] \ No newline at end of file diff --git a/backend/api/migrations/0015_add_admin_adjustment_20230808_1803.py b/backend/api/migrations/0015_add_admin_adjustment_20230808_1803.py new file mode 100644 index 000000000..a63c19022 --- /dev/null +++ b/backend/api/migrations/0015_add_admin_adjustment_20230808_1803.py @@ -0,0 +1,34 @@ +from django.db import migrations + +# Forward operation: Adds the 'Administrative Adjustment' entry +def add_administrative_adjustment(apps, schema_editor): + CreditTradeType = apps.get_model('api', 'CreditTradeType') + credit_trade_type = CreditTradeType( + id=6, + the_type="Administrative Adjustment", + description="An administrative adjustment of the number of Fuel Credits owned by a Fuel Supplier initiated by the BC Government.", + is_gov_only_type=True, + display_order=6, + effective_date='2017-01-01', + expiration_date='2117-01-01' + ) + credit_trade_type.save() + +# Reverse operation: Removes the 'Administrative Adjustment' entry +def remove_administrative_adjustment(apps, schema_editor): + CreditTradeType = apps.get_model('api', 'CreditTradeType') + try: + credit_trade_type = CreditTradeType.objects.get(the_type="Administrative Adjustment") + credit_trade_type.delete() + except CreditTradeType.DoesNotExist: + pass + +class Migration(migrations.Migration): + + dependencies = [ + ('api', '0014_new_label_changes'), + ] + + operations = [ + migrations.RunPython(add_administrative_adjustment, remove_administrative_adjustment) + ] diff --git a/backend/api/migrations/0016_alter_credittrade_number_of_credits.py b/backend/api/migrations/0016_alter_credittrade_number_of_credits.py new file mode 100644 index 000000000..d5bfb4b02 --- /dev/null +++ b/backend/api/migrations/0016_alter_credittrade_number_of_credits.py @@ -0,0 +1,18 @@ +# Generated by Django 3.2.19 on 2023-08-09 00:06 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('api', '0015_add_admin_adjustment_20230808_1803'), + ] + + operations = [ + migrations.AlterField( + model_name='credittrade', + name='number_of_credits', + field=models.IntegerField(), + ), + ] diff --git a/backend/api/migrations/0017_alter_compliace_report_history_status.py b/backend/api/migrations/0017_alter_compliace_report_history_status.py new file mode 100644 index 000000000..04f6c1ec5 --- /dev/null +++ b/backend/api/migrations/0017_alter_compliace_report_history_status.py @@ -0,0 +1,20 @@ +from django.db import migrations, models, connection + +class Migration(migrations.Migration): + + dependencies = [ + ('api', '0016_alter_credittrade_number_of_credits'), + ] + # apply migration only to test database + if connection.settings_dict['NAME'] == 'test_tfrs': + operations = [ + migrations.RemoveField( + model_name='compliancereporthistory', + name='status' + ), + migrations.AddField( + model_name='compliancereporthistory', + name='status', + field=models.ForeignKey(on_delete=models.deletion.PROTECT, related_name='history_records', to='api.compliancereportworkflowstate'), + ), + ] diff --git a/backend/api/migrations/0018_report_history_grouping.py b/backend/api/migrations/0018_report_history_grouping.py new file mode 100644 index 000000000..4373c524f --- /dev/null +++ b/backend/api/migrations/0018_report_history_grouping.py @@ -0,0 +1,58 @@ +from django.db import migrations, transaction, models +import collections + +def update_report_fields(apps, schema_editor): + ComplianceReport = apps.get_model('api', 'compliancereport') + for report in ComplianceReport.objects.filter(supplements__isnull=False): + with transaction.atomic(): + ancestor = report + root = None + latest = None + while ancestor.supplements is not None: + ancestor = ancestor.supplements + + visited = [] + id_traversal = {} + to_visit = collections.deque([ancestor.id]) + i = 0 + + while len(to_visit) > 0: + current_id = to_visit.popleft() + + # break loops + if current_id in visited: + continue + visited.append(current_id) + + current = ComplianceReport.objects.get(id=current_id) + + if current.supplements is None: + root = current + latest = current + # don't count non-supplement reports (really should just be the root) + if current.supplements is not None and \ + not current.status.fuel_supplier_status_id == "Deleted": + latest = current + i += 1 + id_traversal[current_id] = i + for descendant in current.supplemental_reports.order_by('create_timestamp').all(): + to_visit.append(descendant.id) + + for compliance_id, traversal in id_traversal.items(): + ComplianceReport.objects.filter(id=int(compliance_id)) \ + .update(latest_report=latest, root_report=root, traversal=traversal) + + +class Migration(migrations.Migration): + dependencies = [ + ('api', '0017_alter_compliace_report_history_status'), + ] + + operations = [ + migrations.AlterField( + model_name='compliancereport', + name='traversal', + field=models.IntegerField(default=0), + ), + migrations.RunPython(update_report_fields, reverse_code=migrations.RunPython.noop), + ] diff --git a/backend/api/migrations/0019_update_signing_authority_declaration_statement.py b/backend/api/migrations/0019_update_signing_authority_declaration_statement.py new file mode 100644 index 000000000..31781ff0f --- /dev/null +++ b/backend/api/migrations/0019_update_signing_authority_declaration_statement.py @@ -0,0 +1,38 @@ +import logging +from django.db import migrations + +def update_sign_auth_assertion(apps, schema_editor): + """ + Updates the signing authority declaration statement + + Previous label: + "I confirm that records evidencing each matter reported under section 11.11 (2) of the + Regulation are available on request." + + New label: + "I confirm that records evidencing each matter reported under section 17 of the Low Carbon + Fuel (General) Regulation are available on request." + """ + signing_authority_assertion = apps.get_model('api', 'SigningAuthorityAssertion') + try: + assertion = signing_authority_assertion.objects.get(id=1) + assertion.description = ( + 'I confirm that records evidencing each matter reported under section 17 ' + 'of the Low Carbon Fuel (General) Regulation are available on request.' + ) + assertion.save() + except signing_authority_assertion.DoesNotExist: + logging.warning('Failed to update SigningAuthorityAssertion: No entry found with id "1".') + raise + +class Migration(migrations.Migration): + """ + Attaches the update function to the migration operations + """ + dependencies = [ + ('api', '0018_report_history_grouping'), + ] + + operations = [ + migrations.RunPython(update_sign_auth_assertion), + ] diff --git a/backend/api/migrations/0020_correct_effective_date_of_transfer_2095.py b/backend/api/migrations/0020_correct_effective_date_of_transfer_2095.py new file mode 100644 index 000000000..fbb0d2b99 --- /dev/null +++ b/backend/api/migrations/0020_correct_effective_date_of_transfer_2095.py @@ -0,0 +1,39 @@ +import logging +from django.db import migrations, transaction +from django.utils import timezone + +def update_transfer_effective_date(apps, schema_editor): + """ + Update transfer ID #2095 to correct effective date from March 30, 2022 to March 30, 2023. + If any record is not updated, all changes are reverted. + """ + credit_trade_history = apps.get_model('api', 'CreditTradeHistory') + new_trade_effective_date = timezone.datetime.strptime('2023-03-30', "%Y-%m-%d").date() + + # IDs of the CreditTradeHistory records to update + history_ids = [4666, 4709] + + with transaction.atomic(): + for history_id in history_ids: + try: + history = credit_trade_history.objects.get(id=history_id) + history.trade_effective_date = new_trade_effective_date + history.save() + except credit_trade_history.DoesNotExist: + logging.warning( + 'Failed to update CreditTradeHistory: No entry found with id "%s"; ' + 'all changes within this transaction will be reverted.', + history_id + ) + +class Migration(migrations.Migration): + """ + Attaches the update function to the migration operations + """ + dependencies = [ + ('api', '0019_update_signing_authority_declaration_statement'), + ] + + operations = [ + migrations.RunPython(update_transfer_effective_date, reverse_code=migrations.RunPython.noop), + ] diff --git a/backend/api/migrations/0021_correct_compliance_period_of_transfer_2095.py b/backend/api/migrations/0021_correct_compliance_period_of_transfer_2095.py new file mode 100644 index 000000000..fce3d3880 --- /dev/null +++ b/backend/api/migrations/0021_correct_compliance_period_of_transfer_2095.py @@ -0,0 +1,40 @@ +import logging +from django.db import migrations, transaction + +def update_credit_trade_history(apps, schema_editor): + """ + Update transfer ID #2095 to correct compliance period to the year 2023 (compliance_period_id=13) + from the previous year 2022 (compliance_period_id=12) + + If any record is not updated, all changes are reverted. + """ + credit_trade_history = apps.get_model('api', 'CreditTradeHistory') + new_compliance_period_id = 13 + + # IDs of the CreditTradeHistory records to update + history_ids = [978, 979] + + with transaction.atomic(): + for history_id in history_ids: + try: + history = credit_trade_history.objects.get(id=history_id) + history.compliance_period_id = new_compliance_period_id + history.save() + except credit_trade_history.DoesNotExist: + logging.warning( + 'Failed to update CreditTradeHistory: No entry found with id "%s"; ' + 'all changes within this transaction will be reverted.', + history_id + ) + +class Migration(migrations.Migration): + """ + Attaches the update function to the migration operations + """ + dependencies = [ + ('api', '0020_correct_effective_date_of_transfer_2095'), + ] + + operations = [ + migrations.RunPython(update_credit_trade_history, reverse_code=migrations.RunPython.noop), + ] diff --git a/backend/api/models/ComplianceReport.py b/backend/api/models/ComplianceReport.py index cd9c99199..67de36066 100644 --- a/backend/api/models/ComplianceReport.py +++ b/backend/api/models/ComplianceReport.py @@ -233,7 +233,7 @@ class ComplianceReport(Auditable): related_name='latest_reports') traversal = models.IntegerField( - default=1, + default=0, db_comment="Traversal position of this compliance report. " ) @@ -247,7 +247,6 @@ class ComplianceReport(Auditable): db_comment='An explanatory note required when submitting a supplemental report' ) - @property def generated_nickname(self): """ Used for display in the UI when no nickname is set""" diff --git a/backend/api/models/CreditTrade.py b/backend/api/models/CreditTrade.py index b5bac5b9f..3fc1546bf 100644 --- a/backend/api/models/CreditTrade.py +++ b/backend/api/models/CreditTrade.py @@ -70,7 +70,6 @@ class CreditTrade(Auditable): related_name='credit_trades', on_delete=models.PROTECT) number_of_credits = models.IntegerField( - validators=[validators.CreditTradeNumberOfCreditsValidator], db_comment="Number of credits to be transferred on approval" ) fair_market_value_per_credit = models.DecimalField( @@ -132,8 +131,8 @@ def credits_from(self): And for type: Buy and Retirement Credits From is the Respondent """ - # 3 and 5 is government - if self.type.id in [1, 3, 5]: + # 3, 5 and 6 is government + if self.type.id in [1, 3, 5, 6]: return self.initiator # elif self.type.id in [2, 4] return self.respondent @@ -247,6 +246,10 @@ def comment(self): def comment(self, comment): self._comment = comment + def clean(self): + super().clean() + validators.CreditTradeNumberOfCreditsValidator(self.number_of_credits, self) + class Meta: db_table = 'credit_trade' diff --git a/backend/api/models/CreditTradeType.py b/backend/api/models/CreditTradeType.py index ce34072c8..85e3cabef 100644 --- a/backend/api/models/CreditTradeType.py +++ b/backend/api/models/CreditTradeType.py @@ -81,5 +81,27 @@ def friendly_name(self): if self.the_type == "Credit Validation": return "Validation" + + if self.the_type == "Administrative Adjustment": + return "Admin Adjustment" + + return self.the_type + + @property + def notification_name(self): + """ + Front-end Notification language for the Credit Trade Type + """ + if self.the_type in ["Buy", "Sell"]: + return "Transfer" + + if self.the_type == "Credit Reduction" or self.the_type == "Credit Validation": + return "Assessment" + + if self.the_type == "Part 3 Award": + return "Initiative Agreement" + + if self.the_type == "Administrative Adjustment": + return "Admin Adjustment" return self.the_type diff --git a/backend/api/serializers/ComplianceReport.py b/backend/api/serializers/ComplianceReport.py index e53964bdb..93c459fea 100644 --- a/backend/api/serializers/ComplianceReport.py +++ b/backend/api/serializers/ComplianceReport.py @@ -53,6 +53,7 @@ from api.serializers.constants import ComplianceReportValidation from api.services.ComplianceReportService import ComplianceReportService from api.services.OrganizationService import OrganizationService +from api.services.ComplianceReportSummaryService import ComplianceReportSummaryService class ComplianceReportBaseSerializer: def get_last_accepted_offset(self, obj): @@ -473,7 +474,7 @@ def get_deltas(self, obj): ) if qs.exists(): - ancestor_snapshot = qs.first().snapshot + ancestor_snapshot = ComplianceReportDetailSerializer.build_compliance_units(qs.first().snapshot, obj) if int(obj.compliance_period.description) > 2022 else qs.first().snapshot ancestor_computed = False else: # no snapshot. make one. @@ -489,7 +490,7 @@ def get_deltas(self, obj): ) if qs.exists(): - current_snapshot = qs.first().snapshot + current_snapshot = ComplianceReportDetailSerializer.build_compliance_units(qs.first().snapshot, obj) if int(obj.compliance_period.description) > 2022 else qs.first().snapshot else: # no snapshot ser = ComplianceReportDetailSerializer( @@ -517,6 +518,68 @@ def get_deltas(self, obj): current = current.supplements return deltas + + @staticmethod + def build_compliance_units(snapshot, obj): + lines = snapshot['summary']['lines'] + if lines.get('29A') is None: + previous_transactions = [] + previous_snapshots = [] + current = obj + is_supplemental = False + + if current.supplements: + is_supplemental = True + + available_compliance_unit_balance = OrganizationService.get_max_credit_offset_for_interval( + obj.organization, + obj.update_timestamp + ) + net_compliance_unit_balance = int(lines['25']) + desired_net_credit_balance_change = Decimal(0.0) + if is_supplemental: + while current.supplements is not None: + current = current.supplements + if current.credit_transaction is not None: + previous_transactions.append(current.credit_transaction) + if current.compliance_report_snapshot is not None: + previous_snapshots.append(current.compliance_report_snapshot.snapshot) + + total_previous_reduction = Decimal(0.0) + total_previous_validation = Decimal(0.0) + + for transaction in previous_transactions: + if transaction.type.the_type in ['Credit Validation']: + total_previous_validation += transaction.number_of_credits + if transaction.type.the_type in ['Credit Reduction']: + total_previous_reduction += transaction.number_of_credits + desired_net_credit_balance_change = Decimal(lines['25']) + net_compliance_unit_balance = desired_net_credit_balance_change - \ + (total_previous_validation - total_previous_reduction) + + adjusted_balance = available_compliance_unit_balance + net_compliance_unit_balance + if available_compliance_unit_balance <= 0 and net_compliance_unit_balance < 0: + lines['28'] = int((adjusted_balance * Decimal('-600.00')).max(Decimal(0))) if (adjusted_balance < 0) else 0 + lines['29A'] = 0 + total_previous_compliance_units = Decimal(0.0) + for snapshots in previous_snapshots: + if snapshots.get("summary").get("lines") is not None: + total_previous_compliance_units += Decimal(snapshots.get("summary").get("lines").get("25")) + lines['29B'] = Decimal(lines['25']) - total_previous_compliance_units + lines['29C'] = 0 + else: + lines['29A'] = available_compliance_unit_balance + lines['28'] = 0 + if (net_compliance_unit_balance < 0 <= adjusted_balance) or (net_compliance_unit_balance >= 0): + lines['29B'] = net_compliance_unit_balance + elif net_compliance_unit_balance < 0 and adjusted_balance < 0: + lines['29B'] = net_compliance_unit_balance if (adjusted_balance > 0) else -available_compliance_unit_balance + lines['28'] = int((adjusted_balance * Decimal('-600.00')).max(Decimal(0))) if (adjusted_balance < 0) else 0 + lines['29C'] = lines['29A'] + lines['29B'] + snapshot['summary']['total_payable'] = Decimal(lines['11']) + Decimal(lines['22']) + lines['28'] + snapshot['summary']['lines'] = lines + + return snapshot def get_max_credit_offset(self, obj): max_credit_offset = OrganizationService.get_max_credit_offset( @@ -542,150 +605,17 @@ def get_max_credit_offset_exclude_reserved(self, obj): return max_credit_offset_exclude_reserved def get_summary(self, obj): - total_petroleum_diesel = Decimal(0) - total_petroleum_gasoline = Decimal(0) - total_renewable_diesel = Decimal(0) - total_renewable_gasoline = Decimal(0) - total_credits = Decimal(0) - total_debits = Decimal(0) - net_gasoline_class_transferred = Decimal(0) - net_diesel_class_transferred = Decimal(0) - - lines = {} - - if obj.summary is not None: - lines['6'] = obj.summary.gasoline_class_retained \ - if obj.summary.gasoline_class_retained is not None \ - else Decimal(0) - lines['7'] = obj.summary.gasoline_class_previously_retained \ - if obj.summary.gasoline_class_previously_retained is not None \ - else Decimal(0) - lines['8'] = obj.summary.gasoline_class_deferred \ - if obj.summary.gasoline_class_deferred is not None \ - else Decimal(0) - lines['9'] = obj.summary.gasoline_class_obligation \ - if obj.summary.gasoline_class_obligation is not None \ - else Decimal(0) - lines['17'] = obj.summary.diesel_class_retained \ - if obj.summary.diesel_class_retained is not None \ - else Decimal(0) - lines['18'] = obj.summary.diesel_class_previously_retained \ - if obj.summary.diesel_class_previously_retained is not None \ - else Decimal(0) - lines['19'] = obj.summary.diesel_class_deferred \ - if obj.summary.diesel_class_deferred is not None \ - else Decimal(0) - lines['20'] = obj.summary.diesel_class_obligation \ - if obj.summary.diesel_class_obligation is not None \ - else Decimal(0) - lines['26'] = Decimal(obj.summary.credits_offset) \ - if obj.summary.credits_offset is not None else Decimal(0) - lines['26A'] = Decimal(obj.summary.credits_offset_a) \ - if obj.summary.credits_offset_a is not None else Decimal(0) - lines['26B'] = Decimal(obj.summary.credits_offset_b) \ - if obj.summary.credits_offset_b is not None else Decimal(0) - lines['26C'] = Decimal(obj.summary.credits_offset_c) \ - if obj.summary.credits_offset_c is not None else Decimal(0) - else: - lines['6'] = Decimal(0) - lines['7'] = Decimal(0) - lines['8'] = Decimal(0) - lines['9'] = Decimal(0) - lines['17'] = Decimal(0) - lines['18'] = Decimal(0) - lines['19'] = Decimal(0) - lines['20'] = Decimal(0) - lines['26'] = Decimal(0) - lines['26A'] = Decimal(0) - lines['26B'] = Decimal(0) - lines['26C'] = Decimal(0) - - if obj.schedule_a: - net_gasoline_class_transferred += \ - obj.schedule_a.net_gasoline_class_transferred - net_diesel_class_transferred += \ - obj.schedule_a.net_diesel_class_transferred - - lines['5'] = net_gasoline_class_transferred - lines['16'] = net_diesel_class_transferred - - if obj.schedule_b: - total_petroleum_diesel += obj.schedule_b.total_petroleum_diesel - total_petroleum_gasoline += obj.schedule_b.total_petroleum_gasoline - total_renewable_diesel += obj.schedule_b.total_renewable_diesel - total_renewable_gasoline += obj.schedule_b.total_renewable_gasoline - total_credits += obj.schedule_b.total_credits - total_debits += obj.schedule_b.total_debits - - if obj.schedule_c: - total_petroleum_diesel += obj.schedule_c.total_petroleum_diesel - total_petroleum_gasoline += obj.schedule_c.total_petroleum_gasoline - total_renewable_diesel += obj.schedule_c.total_renewable_diesel - total_renewable_gasoline += obj.schedule_c.total_renewable_gasoline - - lines['1'] = total_petroleum_gasoline - lines['2'] = total_renewable_gasoline - lines['3'] = lines['1'] + lines['2'] - lines['4'] = (lines['3'] * Decimal('0.05')).quantize( - Decimal('1.'), rounding=ROUND_HALF_UP - ) # hardcoded 5% renewable requirement - lines['10'] = lines['2'] + lines['5'] - lines['6'] + lines['7'] + \ - lines['8'] - lines['9'] - lines['11'] = ((lines['4'] - lines['10']) * Decimal('0.30')).max( - Decimal(0)).quantize(Decimal('.01'), rounding=ROUND_HALF_UP) - - lines['12'] = total_petroleum_diesel - lines['13'] = total_renewable_diesel - lines['14'] = lines['12'] + lines['13'] - lines['15'] = (lines['14'] * Decimal('0.04')).quantize( - Decimal('1.'), rounding=ROUND_HALF_UP - ) # hardcoded 4% renewable requirement - lines['21'] = lines['13'] + lines['16'] - lines['17'] + lines['18'] + \ - lines['19'] - lines['20'] - lines['22'] = ((lines['15'] - lines['21']) * Decimal('0.45')).max( - Decimal(0)).quantize(Decimal('.01'), rounding=ROUND_HALF_UP) - - lines['23'] = total_credits - lines['24'] = total_debits - lines['25'] = lines['23'] - lines['24'] - - # if current_balance is positive it means the supplier - # has a positive amount of credits for this compliance period - # and there is no penalty, otherwise use current_balance - # to calculate penalty - current_balance = lines['25'] + lines['26'] - if current_balance > 0: - lines['27'] = 0 - else: - lines['27'] = current_balance - - # 26C represents credits that need to be returned to the fuel supplier. - # Line 27 should end up being zero in this situation because - # 26C is the difference between lines 26A and 25 when 26A > 25 - if lines['26C'] is not None and lines['26C'] > 0: - lines['27'] = 0 # eqv. to lines['25'] + lines['26A'] - lines['26C'] - - # Penalty adjustment made by business area for - # 2023 and above compliance periods - if int(obj.compliance_period.description) <= 2022: - lines['28'] = (lines['27'] * Decimal('-200.00')).max(Decimal(0)) - else: - lines['28'] = (lines['27'] * Decimal('-600.00')).max(Decimal(0)) - - total_payable = lines['11'] + lines['22'] + lines['28'] - - synthetic_totals = { - "total_petroleum_diesel": total_petroleum_diesel, - "total_petroleum_gasoline": total_petroleum_gasoline, - "total_renewable_diesel": total_renewable_diesel, - "total_renewable_gasoline": total_renewable_gasoline, - "net_diesel_class_transferred": net_diesel_class_transferred, - "net_gasoline_class_transferred": net_gasoline_class_transferred, - "lines": lines, - "total_payable": total_payable - } + """ + Retrieve a summary that merges synthetic totals with existing summary data. + + :param obj: The compliance report object containing summary and synthetic details. + :return: A dictionary combining synthetic totals with existing summary data. + """ + # Compute the synthetic totals for the provided compliance report object. + synthetic_totals = ComplianceReportSummaryService.calculate_synthetic_totals(obj) - if obj.summary is not None: + # If a summary already exists for the object, merge it with the computed synthetic totals. + if obj.summary: ser = ScheduleSummaryDetailSerializer(obj.summary) data = ser.data synthetic_totals = {**data, **synthetic_totals} @@ -1250,6 +1180,8 @@ def create(self, validated_data): ComplianceReport.objects.filter(root_report=root_report).update(latest_report=new_compliance_report) else: new_compliance_report.traversal = previous_report.traversal + ComplianceReport.objects.filter(root_report_id=root_report.id)\ + .update(latest_report=new_compliance_report) else: new_compliance_report.root_report = new_compliance_report new_compliance_report.latest_report = new_compliance_report @@ -1292,6 +1224,7 @@ class ComplianceReportUpdateSerializer( ) actions = serializers.SerializerMethodField() actor = serializers.SerializerMethodField() + deltas = serializers.SerializerMethodField() display_name = SerializerMethodField() max_credit_offset = SerializerMethodField() max_credit_offset_exclude_reserved = SerializerMethodField() @@ -1303,6 +1236,7 @@ class ComplianceReportUpdateSerializer( strip_summary = False disregard_status = False + skip_deltas = False def get_display_name(self, obj): if obj.nickname is not None and obj.nickname != '': @@ -1372,6 +1306,69 @@ def get_history(self, obj): return serializer.data else: return None + + def get_deltas(self, obj): + + deltas = [] + + if self.skip_deltas: + return deltas + + current = obj + + while current: + if current.supplements: + ancestor = current.supplements + + qs = ComplianceReportSnapshot.objects.filter( + compliance_report=ancestor + ) + + if qs.exists(): + ancestor_snapshot = ComplianceReportDetailSerializer.build_compliance_units(qs.first().snapshot, obj) if int(obj.compliance_period.description) > 2022 else qs.first().snapshot + ancestor_computed = False + else: + # no snapshot. make one. + ser = ComplianceReportDetailSerializer( + ancestor, context=self.context + ) + ser.skip_deltas = True + ancestor_snapshot = ser.data + ancestor_computed = True + + qs = ComplianceReportSnapshot.objects.filter( + compliance_report=current + ) + + if qs.exists(): + current_snapshot = ComplianceReportDetailSerializer.build_compliance_units(qs.first().snapshot, obj) if int(obj.compliance_period.description) > 2022 else qs.first().snapshot + else: + # no snapshot + ser = ComplianceReportDetailSerializer( + current, context=self.context + ) + ser.skip_deltas = True + current_snapshot = ser.data + + deltas += [{ + 'levels_up': 1, + 'ancestor_id': ancestor.id, + 'ancestor_display_name': ancestor.nickname + if (ancestor.nickname is not None and + ancestor.nickname != '') + else ancestor.generated_nickname, + 'delta': ComplianceReportService.compute_delta( + current_snapshot, ancestor_snapshot + ), + 'snapshot': { + 'data': ancestor_snapshot, + 'computed': ancestor_computed + } + }] + + current = current.supplements + + return deltas def update(self, instance, validated_data): request = self.context.get('request') @@ -1390,19 +1387,20 @@ def update(self, instance, validated_data): instance.compliance_period.description ) - if summary_data and instance.supplements_id is None and \ - summary_data.get('credits_offset', 0) and \ - summary_data.get('credits_offset', 0) > max_credit_offset: - raise (serializers.ValidationError( - 'Insufficient available credit balance. Please adjust Line 26.' - )) - - if summary_data and instance.supplements_id and \ - summary_data.get('credits_offset_b', 0) and \ - summary_data.get('credits_offset_b', 0) > max_credit_offset and not self.strip_summary: - raise (serializers.ValidationError( - 'Insufficient available credit balance. Please adjust Line 26b.' - )) + if int(instance.compliance_period.description) <= 2022: + if summary_data and instance.supplements_id is None and \ + summary_data.get('credits_offset', 0) and \ + summary_data.get('credits_offset', 0) > max_credit_offset: + raise (serializers.ValidationError( + 'Insufficient available credit balance. Please adjust Line 26.' + )) + + if summary_data and instance.supplements_id and \ + summary_data.get('credits_offset_b', 0) and \ + summary_data.get('credits_offset_b', 0) > max_credit_offset and not self.strip_summary: + raise (serializers.ValidationError( + 'Insufficient available credit balance. Please adjust Line 26b.' + )) if 'status' in validated_data: status_data = validated_data.pop('status') @@ -1679,7 +1677,7 @@ class Meta: fields = ( 'status', 'type', 'compliance_period', 'organization', 'schedule_a', 'schedule_b', 'schedule_c', 'schedule_d', - 'summary', 'read_only', 'has_snapshot', 'actions', 'actor', + 'summary', 'read_only', 'has_snapshot', 'actions', 'actor', 'deltas', 'display_name', 'supplemental_note', 'is_supplemental', 'max_credit_offset', 'max_credit_offset_exclude_reserved', 'total_previous_credit_reductions', 'supplemental_number', 'last_accepted_offset', 'history', @@ -1717,6 +1715,8 @@ def destroy(self): compliance_report.status.fuel_supplier_status = \ ComplianceReportStatus.objects.get(status="Deleted") + ComplianceReport.objects.filter(root_report_id=compliance_report.root_report.id)\ + .update(latest_report=compliance_report.supplements) compliance_report.status.save() class Meta: diff --git a/backend/api/serializers/CreditTrade.py b/backend/api/serializers/CreditTrade.py index 79a8bb581..0af953064 100644 --- a/backend/api/serializers/CreditTrade.py +++ b/backend/api/serializers/CreditTrade.py @@ -48,10 +48,8 @@ from .Organization import OrganizationMinSerializer, OrganizationSerializer from .User import UserMinSerializer -INSUFFICIENT_CREDITS_MESSAGE = "Unable to initiate this Credit Transfer " \ - "Proposal. Your organization either does not have enough " \ - "validated credits or has pending Credit Transfer Proposal(s) that " \ - "could result in an insufficient credit balance for this transfer." +INSUFFICIENT_CREDITS_MESSAGE = "Unable to initiate transfer. " \ + "Your organization does not have enough compliance units for this transfer." class CreditTradeCreateSerializer(serializers.ModelSerializer): @@ -135,7 +133,8 @@ def validate(self, data): the_type__in=[ "Credit Validation", "Credit Reduction", - "Part 3 Award" + "Part 3 Award", + "Administrative Adjustment" ] ).only('id') ) @@ -260,7 +259,7 @@ class Meta: extra_kwargs = { 'compliance_period': { 'error_messages': { - 'does_not_exist': "Please specify the Compliance Period " + 'does_not_exist': "Please specify the compliance period " "in which the transaction relates." } }, @@ -632,7 +631,7 @@ class Meta: extra_kwargs = { 'compliance_period': { 'error_messages': { - 'does_not_exist': "Please specify the Compliance Period " + 'does_not_exist': "Please specify the compliance period " "in which the transaction relates." } }, diff --git a/backend/api/serializers/CreditTradeType.py b/backend/api/serializers/CreditTradeType.py index 27beb6e4c..851c90def 100644 --- a/backend/api/serializers/CreditTradeType.py +++ b/backend/api/serializers/CreditTradeType.py @@ -31,7 +31,7 @@ class Meta: fields = ( 'id', 'the_type', 'description', 'effective_date', 'expiration_date', - 'display_order', 'is_gov_only_type') + 'display_order', 'is_gov_only_type', 'notification_name') class CreditTradeTypeMinSerializer(serializers.ModelSerializer): diff --git a/backend/api/serializers/Document.py b/backend/api/serializers/Document.py index 7a5a4d9e2..962fbb04c 100644 --- a/backend/api/serializers/Document.py +++ b/backend/api/serializers/Document.py @@ -127,7 +127,7 @@ def validate_title(self, value): the_type="Evidence").id: if not value: raise serializers.ValidationError( - "Please provide the name of the Part 3 Agreement to which " + "Please provide the name of the Initiative Agreement to which " "the submission relates." ) @@ -150,8 +150,8 @@ def validate_milestone(self, value): the_type="Evidence").id: if not value: raise serializers.ValidationError( - "Please indicate the Milestone(s) to which the submission " - "relates." + "Please indicate the Designated Action(s) to which the " + "submission relates." ) return value @@ -223,8 +223,8 @@ class Meta: extra_kwargs = { 'compliance_period': { 'error_messages': { - 'does_not_exist': "Please specify the Compliance Period " - "to which the request relates." + 'does_not_exist': "Please specify the compliance period " + "to which the submission relates." } } } @@ -432,7 +432,7 @@ def validate_title(self, value): if document.type == DocumentType.objects.get(the_type="Evidence").id: if not value: raise serializers.ValidationError( - "Please provide the name of the Part 3 Agreement to which " + "Please provide the name of the Initiative Agreement to which " "the submission relates." ) @@ -455,8 +455,8 @@ def validate_milestone(self, value): the_type="Evidence"): if not value: raise serializers.ValidationError( - "Please indicate the Milestone(s) to which the submission " - "relates." + "Please indicate the Designated Action(s) to which the " + "submission relates." ) return value @@ -516,8 +516,8 @@ def validate(self, data): if 'milestone' in request.data and \ not request.data.get('milestone'): raise serializers.ValidationError({ - 'milestone': "Please indicate the Milestone(s) to " - "which the submission relates." + 'milestone': "Please indicate the Designated " + "Action(s) to which the submission relates." }) current_attachments = document.attachments @@ -656,8 +656,8 @@ class Meta: extra_kwargs = { 'compliance_period': { 'error_messages': { - 'does_not_exist': "Please specify the Compliance Period " - "to which the request relates." + 'does_not_exist': "Please specify the compliance period " + "to which the submission relates." } }, 'milestone': { diff --git a/backend/api/services/ComplianceReportService.py b/backend/api/services/ComplianceReportService.py index b18015228..38f3c05b0 100644 --- a/backend/api/services/ComplianceReportService.py +++ b/backend/api/services/ComplianceReportService.py @@ -255,6 +255,7 @@ def create_director_transactions(compliance_report, creating_user): raise InvalidStateException() snapshot = compliance_report.snapshot + COMPLIANCE_PERIOD_2023_AND_ABOVE = int(snapshot['compliance_period']['description']) >= 2023 if 'summary' not in snapshot: raise InvalidStateException() @@ -262,33 +263,45 @@ def create_director_transactions(compliance_report, creating_user): raise InvalidStateException() lines = snapshot['summary']['lines'] - - desired_net_credit_balance_change = Decimal(0.0) - - if Decimal(lines['25']) > Decimal(0): - desired_net_credit_balance_change = Decimal(lines['25']) - elif Decimal(lines['25']) < 0 and Decimal(lines['26']) > Decimal(0): - desired_net_credit_balance_change = Decimal(lines['26']) * Decimal(-1.0) - - required_credit_transaction = desired_net_credit_balance_change - \ - (total_previous_validation - total_previous_reduction) - - if settings.DEVELOPMENT: - print('line 25 of current report: {}'.format(lines['25'])) - print('desired credit balance change: {}'.format(desired_net_credit_balance_change)) - print('required transaction to effect change: {}'.format(required_credit_transaction)) - - if is_supplemental and Decimal(lines['25']) < 0 and \ - (Decimal(lines['26']) + Decimal(lines['25'])) > 0: - required_credit_transaction = Decimal(lines['26']) + Decimal(lines['25']) - - # Code 26C is used to identify credits that must be refunded to the supplier. - # This occurs when our debit position decreases and we have already spent credits. - # In such cases, any excess credits must be returned to the supplier. - if is_supplemental and Decimal(lines.get('26C', 0)) > 0: - print("*** DIRECTOR 26C Increase to Credits ***") - required_credit_transaction = Decimal(lines['26C']) - + if COMPLIANCE_PERIOD_2023_AND_ABOVE: + # If a compliance report is in a deficit position(i.e., a negative net compliance unit balance for the + # compliance period or negative value in Line 25 in the summary section) that is greater than + # the organization’s available credit balance, then the available compliance unit balance needs to be + # zeroed out when the director assesses (accepts) the compliance report. + if Decimal(lines['25']) < Decimal(0) \ + and (Decimal(lines['29A']) + Decimal(lines['25'])) < 0: + required_credit_transaction = Decimal(lines['29A']) * Decimal(-1.0) + else: + desired_net_credit_balance_change = Decimal(lines['25']) + required_credit_transaction = desired_net_credit_balance_change - \ + (total_previous_validation - total_previous_reduction) + else: + desired_net_credit_balance_change = Decimal(0.0) + + if Decimal(lines['25']) > Decimal(0): + desired_net_credit_balance_change = Decimal(lines['25']) + elif Decimal(lines['25']) < 0 and Decimal(lines['26']) > Decimal(0): + desired_net_credit_balance_change = Decimal(lines['26']) * Decimal(-1.0) + + required_credit_transaction = desired_net_credit_balance_change - \ + (total_previous_validation - total_previous_reduction) + + if settings.DEVELOPMENT: + print('line 25 of current report: {}'.format(lines['25'])) + print('desired credit balance change: {}'.format(desired_net_credit_balance_change)) + print('required transaction to effect change: {}'.format(required_credit_transaction)) + + if is_supplemental and Decimal(lines['25']) < 0 and \ + (Decimal(lines['26']) + Decimal(lines['25'])) > 0: + required_credit_transaction = Decimal(lines['26']) + Decimal(lines['25']) + + # Code 26C is used to identify credits that must be refunded to the supplier. + # This occurs when our debit position decreases and we have already spent credits. + # In such cases, any excess credits must be returned to the supplier. + if is_supplemental and Decimal(lines['26C']) > 0: + print("*** DIRECTOR 26C Increase to Credits ***") + required_credit_transaction = Decimal(lines['26C']) + if required_credit_transaction > Decimal(0): # do validation for Decimal(lines['25']) credit_transaction = CreditTrade( @@ -309,7 +322,19 @@ def create_director_transactions(compliance_report, creating_user): CreditTradeService.pvr_notification(None, credit_transaction) else: if required_credit_transaction < Decimal(0): - # do_reduction for Decimal(lines['26']) + if COMPLIANCE_PERIOD_2023_AND_ABOVE: + # Fetch the organization's balance from organization_balance property + org_balance = Decimal(compliance_report.organization.organization_balance['validated_credits']) + + # Deduct the pending deductions, if any, from the organization balance. + if 'deductions' in compliance_report.organization.organization_balance: + org_balance -= Decimal(compliance_report.organization.organization_balance['deductions']) + + # If required_credit_transaction is more negative than the organization balance, + # set it to be equal to the organization balance. + if org_balance + required_credit_transaction < Decimal(0): + required_credit_transaction = -org_balance + credit_transaction = CreditTrade( initiator=Organization.objects.get(id=1), respondent=compliance_report.organization, diff --git a/backend/api/services/ComplianceReportSpreadSheet.py b/backend/api/services/ComplianceReportSpreadSheet.py index b7ca48e2d..ede04739f 100644 --- a/backend/api/services/ComplianceReportSpreadSheet.py +++ b/backend/api/services/ComplianceReportSpreadSheet.py @@ -84,7 +84,7 @@ def add_schedule_a(self, schedule_a): worksheet.write(row_index, 3, record['transfer_type']) worksheet.write(row_index, 4, Decimal(record['quantity']), quantity_format) - def add_schedule_b(self, schedule_b): + def add_schedule_b(self, schedule_b, compliance_period): worksheet = self.workbook.add_sheet("Schedule B") row_index = 0 @@ -93,6 +93,12 @@ def add_schedule_b(self, schedule_b): "Quantity", "Units", "Carbon Intensity Limit", "Carbon Intensity of Fuel", "Energy Density", "EER", "Energy Content", "Credit", "Debit" ] + if compliance_period >= 2023: + columns = [ + "Fuel Type", "Fuel Class", "Provision", "Fuel Code or Schedule D Provision", + "Quantity", "Units", "Carbon Intensity Limit", "Carbon Intensity of Fuel", + "Energy Density", "EER", "Energy Content", "Compliance Units" + ] header_style = xlwt.easyxf('font: bold on') @@ -135,10 +141,18 @@ def add_schedule_b(self, schedule_b): worksheet.write(row_index, 8, Decimal(record['energy_density'])) worksheet.write(row_index, 9, Decimal(record['eer'])) worksheet.write(row_index, 10, Decimal(record['energy_content'])) - if record['credits'] is not None: - worksheet.write(row_index, 11, Decimal(record['credits'])) - if record['debits'] is not None: - worksheet.write(row_index, 12, Decimal(record['debits'])) + if compliance_period < 2023: + if record['credits'] is not None: + worksheet.write(row_index, 11, Decimal(record['credits'])) + if record['debits'] is not None: + worksheet.write(row_index, 12, Decimal(record['debits'])) + else: + compliance_units = None + if record['credits'] is not None: + compliance_units = Decimal(record['credits']) + if compliance_units is None and record['debits'] is not None: + compliance_units = Decimal(record['debits']) * -1 + worksheet.write(row_index, 11, compliance_units) def add_schedule_c(self, schedule_c): worksheet = self.workbook.add_sheet("Schedule C") @@ -243,7 +257,7 @@ def add_schedule_d(self, schedule_d): worksheet.write(row_index, 0, output['description']) worksheet.write(row_index, 1, Decimal(output['intensity']), value_format) - def add_schedule_summary(self, summary): + def add_schedule_summary(self, summary, compliance_period): worksheet = self.workbook.add_sheet("Summary") row_index = 0 @@ -252,6 +266,8 @@ def add_schedule_summary(self, summary): value_format = xlwt.easyxf(num_format_str='#,##0.00') currency_format = xlwt.easyxf(num_format_str='$#,##0.00') description_format = xlwt.easyxf('align: wrap on') + if summary is None: + return line_details = { '1': 'Volume of gasoline class non-renewable fuel supplied', @@ -290,15 +306,18 @@ def add_schedule_summary(self, summary): '27': 'Outstanding debit balance', '28': 'Part 3 non-compliance penalty payable' } + if compliance_period >= 2023: + line_details['25'] = 'Net compliance unit balance for compliance period' + line_details['29A'] = 'Available compliance unit balance on March 31, ' + str(int(compliance_period) + 1) + line_details['29B'] = 'Compliance unit balance change from assessment' + line_details['29C'] = 'Available compliance unit balance after assessment on March 31, ' + str(int(compliance_period) + 1) + line_details['28'] = 'Non-compliance penalty payable (' + str(int(Decimal(summary['lines']['28'])/600)) + ' units * $600 CAD per unit)' line_format = defaultdict(lambda: quantity_format) line_format['11'] = currency_format line_format['22'] = currency_format line_format['28'] = currency_format - if summary is None: - return - columns = [ "Part 2 Gasoline Class - 5% Renewable Requirement", "Line", @@ -331,7 +350,7 @@ def add_schedule_summary(self, summary): row_index += 1 columns = [ - "Part 3 - Low Carbon Fuel Requirement Summary", + "Part 3 - Low Carbon Fuel Requirement Summary" if compliance_period < 2023 else "Low Carbon Fuel Requirement", "Line", "Value" ] @@ -339,11 +358,21 @@ def add_schedule_summary(self, summary): for col_index, value in enumerate(columns): worksheet.write(row_index, col_index, value, header_style) - for line in range(23, 28+1): - row_index += 1 - worksheet.write(row_index, 0, line_details[str(line)], description_format) - worksheet.write(row_index, 1, 'Line {}'.format(line)) - worksheet.write(row_index, 2, Decimal(summary['lines'][str(line)]), line_format[str(line)]) + if compliance_period >= 2023: + compliance_lines = ['25','29A','29B','28','29C'] + for line in compliance_lines: + if line != '28' or (line == '28' and summary['lines'][line] > 0): + row_index += 1 + worksheet.write(row_index, 0, line_details[line], description_format) + if line.isdigit(): + worksheet.write(row_index, 1, f'Line {line}') + worksheet.write(row_index, 2, Decimal(summary['lines'][line]), line_format[str(line)]) + else: + for line in range(23, 28+1): + row_index += 1 + worksheet.write(row_index, 0, line_details[str(line)], description_format) + worksheet.write(row_index, 1, 'Line {}'.format(line)) + worksheet.write(row_index, 2, Decimal(summary['lines'][str(line)]), line_format[str(line)]) row_index += 1 columns = [ diff --git a/backend/api/services/ComplianceReportSummaryService.py b/backend/api/services/ComplianceReportSummaryService.py new file mode 100644 index 000000000..3b5b22684 --- /dev/null +++ b/backend/api/services/ComplianceReportSummaryService.py @@ -0,0 +1,262 @@ +from decimal import Decimal, ROUND_HALF_UP +from django.db.models import Q +from django.db.transaction import on_commit +from api.services.OrganizationService import OrganizationService + + +class ComplianceReportSummaryService(object): + """ + Helper functions for Compliance Report Summary Calculations + """ + + @staticmethod + def initialize_lines(): + """Initialize all lines to 0""" + return {key: Decimal(0) for key in ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', + '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', + '22', '23', '24', '25', '26', '26A', '26B', '26C', '27', + '28', '29A', '29B', '29C']} + + @staticmethod + def extract_summary_values(summary): + """Extract and assign values from summary to corresponding lines.""" + attributes = [ + ('gasoline_class_retained', '6'), + ('gasoline_class_previously_retained', '7'), + ('gasoline_class_deferred', '8'), + ('gasoline_class_obligation', '9'), + ('diesel_class_retained', '17'), + ('diesel_class_previously_retained', '18'), + ('diesel_class_deferred', '19'), + ('diesel_class_obligation', '20'), + ('credits_offset', '26'), + ('credits_offset_a', '26A'), + ('credits_offset_b', '26B'), + ('credits_offset_c', '26C') + ] + return {line: Decimal(0) if getattr(summary, attr) is None else getattr(summary, attr) for attr, line in attributes} + + @staticmethod + def calculate_synthetic_totals(obj): + """ + Calculate synthetic totals for a given object based on its schedules. + This method takes in a main object and, based on its schedules, calculates the various totals and their + interactions, returning a dictionary that represents the synthetic totals for different fields. + + :param obj: The main object which contains schedules and other relevant attributes. + :return: A dictionary containing the synthetic totals for different fields. + """ + # Initialize the lines with default values + lines = ComplianceReportSummaryService.initialize_lines() + + # If the object has a summary, update the initialized lines with its values + if obj.summary: + lines.update(ComplianceReportSummaryService.extract_summary_values(obj.summary)) + + # Extract values from individual schedules and update the lines accordingly + lines.update(ComplianceReportSummaryService.process_schedule_a(obj.schedule_a, lines)) + lines.update(ComplianceReportSummaryService.process_schedule_b(obj.schedule_b, lines)) + lines.update(ComplianceReportSummaryService.process_schedule_c(obj.schedule_c, lines)) + + # Compute derived gasoline totals + lines['3'] = lines['1'] + lines['2'] # Sum of petroleum and renewable gasoline + # Apply hardcoded 5% renewable requirement to the sum + lines['4'] = (lines['3'] * Decimal('0.05')).quantize(Decimal('1.'), rounding=ROUND_HALF_UP) + + # Sum up adjustments for gasoline values + lines['10'] = lines['2'] + lines['5'] - lines['6'] + lines['7'] + lines['8'] - lines['9'] + lines['11'] = ((lines['4'] - lines['10']) * Decimal('0.30')).max(Decimal(0)).quantize(Decimal('.01'), rounding=ROUND_HALF_UP) + + # Compute derived diesel totals + lines['14'] = lines['12'] + lines['13'] # Sum of petroleum and renewable diesel + # Apply hardcoded 4% renewable requirement to the sum + lines['15'] = (lines['14'] * Decimal('0.04')).quantize(Decimal('1.'), rounding=ROUND_HALF_UP) + # Sum up adjustments for diesel values + lines['21'] = lines['13'] + lines['16'] - lines['17'] + lines['18'] + lines['19'] - lines['20'] + lines['22'] = ((lines['15'] - lines['21']) * Decimal('0.45')).max(Decimal(0)).quantize(Decimal('.01'), rounding=ROUND_HALF_UP) + + # Calculate credits, debits, and resulting balance + lines['25'] = lines['23'] - lines['24'] # Resulting balance from credits minus debits + credit_difference = Decimal(lines['25']) + + # Determine if there's a penalty based on current balance + current_balance = lines['25'] + lines['26'] + lines['27'] = 0 if current_balance > 0 else current_balance + + # Adjust for credits that need to be returned + if lines.get('26C') and lines['26C'] > 0: + lines['27'] = 0 + + # Handle the logic for different compliance periods + if int(obj.compliance_period.description) <= 2022: + lines['28'] = int((lines['27'] * Decimal('-200.00')).max(Decimal(0))) + else: + # For later compliance periods, gather maximum available credit offsets + max_credit_offset = max(0, OrganizationService.get_max_credit_offset( + obj.organization, + obj.compliance_period.description + )) + max_credit_offset_exclude_reserved = max(0, OrganizationService.get_max_credit_offset( + obj.organization, + obj.compliance_period.description, + exclude_reserved=True + )) + if obj.summary is not None and obj.summary.credits_offset is not None and obj.summary.credits_offset > 0: + max_credit_offset += obj.summary.credits_offset + available_compliance_unit_balance = min(max_credit_offset, max_credit_offset_exclude_reserved) + net_compliance_unit_balance = lines['25'] + + # Initialize snapshots and txs to their default values. + previous_snapshots = [] + previous_transactions = [] + + # If there are supplements, fetch and process previous transactions and snapshots + if obj.supplements: + previous_transactions, previous_snapshots = ComplianceReportSummaryService.get_previous_values(obj) + total_previous_validation, total_previous_reduction = ComplianceReportSummaryService.calculate_balance_from_transactions(previous_transactions) + + desired_net_credit_balance_change = Decimal(lines['25']) + net_compliance_unit_balance = desired_net_credit_balance_change - (total_previous_validation - total_previous_reduction) + + adjusted_balance = available_compliance_unit_balance + net_compliance_unit_balance + + # Update the 'lines' dictionary with the newly computed values from 'compute_lines_balance' without overwriting existing entries. + updated_lines = ComplianceReportSummaryService.compute_lines_balance(net_compliance_unit_balance, adjusted_balance, + available_compliance_unit_balance, previous_snapshots, credit_difference) + + lines.update(updated_lines) + + # Compile and return the final synthetic totals + synthetic_totals = { + "total_petroleum_diesel": lines['12'], + "total_petroleum_gasoline": lines['1'], + "total_renewable_diesel": lines['13'], + "total_renewable_gasoline": lines['2'], + "net_diesel_class_transferred": lines['16'], + "net_gasoline_class_transferred": lines['5'], + "lines": lines, + "total_payable": lines['11'] + lines['22'] + lines.get('28', Decimal(0)) # Assuming '28' might be an optional field + } + return synthetic_totals + + @staticmethod + def process_schedule_a(schedule, lines): + """Process schedule A values and return related values.""" + if schedule: + net_gasoline_class_transferred = Decimal(schedule.net_gasoline_class_transferred if schedule else 0) + net_diesel_class_transferred = Decimal(schedule.net_diesel_class_transferred if schedule else 0) + + lines['5'] = net_gasoline_class_transferred + lines['16'] = net_diesel_class_transferred + + return lines + + @staticmethod + def process_schedule_b(schedule, lines): + """Process schedule B values and return related values.""" + if schedule: + total_petroleum_diesel = Decimal(schedule.total_petroleum_diesel) + total_petroleum_gasoline = Decimal(schedule.total_petroleum_gasoline) + total_renewable_diesel = Decimal(schedule.total_renewable_diesel) + total_renewable_gasoline = Decimal(schedule.total_renewable_gasoline) + total_credits = Decimal(schedule.total_credits) + total_debits = Decimal(schedule.total_debits) + + lines['1'] += total_petroleum_gasoline + lines['2'] += total_renewable_gasoline + lines['12'] += total_petroleum_diesel + lines['13'] += total_renewable_diesel + lines['23'] += total_credits + lines['24'] += total_debits + + return lines + + @staticmethod + def process_schedule_c(schedule, lines): + """Process schedule C values and return related values.""" + if schedule: + total_petroleum_diesel = Decimal(schedule.total_petroleum_diesel) + total_petroleum_gasoline = Decimal(schedule.total_petroleum_gasoline) + total_renewable_diesel = Decimal(schedule.total_renewable_diesel) + total_renewable_gasoline = Decimal(schedule.total_renewable_gasoline) + + lines['1'] += total_petroleum_gasoline + lines['2'] += total_renewable_gasoline + lines['12'] += total_petroleum_diesel + lines['13'] += total_renewable_diesel + + return lines + + @staticmethod + def get_previous_values(current): + """ + Traverse through supplements to gather previous transactions and snapshots. + + :param current: The starting object which has potential supplements. + :return: A tuple containing lists of previous transactions and snapshots. + """ + previous_transactions = [] + previous_snapshots = [] + + # Loop through the supplements to fetch transactions and snapshots + while current.supplements: + current = current.supplements + if current.credit_transaction: + previous_transactions.append(current.credit_transaction) + if current.compliance_report_snapshot: + previous_snapshots.append(current.compliance_report_snapshot.snapshot) + + return previous_transactions, previous_snapshots + + @staticmethod + def calculate_balance_from_transactions(transactions): + """ + Calculate the total of previous validations and reductions from the transactions. + + :param transactions: A list of credit transactions. + :return: A tuple containing the total of previous validations and reductions. + """ + # Calculate the total number of credits for validations + total_previous_validation = sum(t.number_of_credits for t in transactions if t.type.the_type == 'Credit Validation') + # Calculate the total number of credits for reductions + total_previous_reduction = sum(t.number_of_credits for t in transactions if t.type.the_type == 'Credit Reduction') + + return total_previous_validation, total_previous_reduction + + @staticmethod + def compute_lines_balance(net_compliance_unit_balance, adjusted_balance, available_compliance_unit_balance, + previous_snapshots, credit_difference): + """ + Compute the balance for various line items based on the given parameters. + + :param net_compliance_unit_balance: Net balance of compliance units. + :param adjusted_balance: Adjusted balance value. + :param available_compliance_unit_balance: Available balance of compliance units excluding reserves. + :param previous_snapshots: List of previous compliance report snapshots. + :return: A dictionary representing the computed balance for different line items. + """ + lines = {} + total_previous_compliance_units = Decimal(0.0) + + # Determine balance for line items based on available and net compliance unit balances + if available_compliance_unit_balance <= 0 and net_compliance_unit_balance < 0: + lines['28'] = int((adjusted_balance * Decimal('-600.00')).max(Decimal(0))) if (adjusted_balance < 0) else 0 + lines['29A'] = 0 + if previous_snapshots: + total_previous_compliance_units = sum(Decimal(snap.get("summary", {}).get("lines", {}).get("25", 0)) for snap in previous_snapshots) + lines['29B'] = 0 if (available_compliance_unit_balance <= 0) else credit_difference - total_previous_compliance_units + lines['29C'] = 0 + else: + lines['29A'] = available_compliance_unit_balance + lines['28'] = 0 + + # Adjust the line item values based on the net and adjusted balances + if net_compliance_unit_balance < 0 and adjusted_balance < 0: + lines['29B'] = net_compliance_unit_balance if adjusted_balance > 0 else -available_compliance_unit_balance + lines['28'] = int((adjusted_balance * Decimal('-600.00')).max(Decimal(0))) if adjusted_balance < 0 else 0 + else: + lines['29B'] = net_compliance_unit_balance + + lines['29C'] = lines['29A'] + lines['29B'] + + return lines diff --git a/backend/api/services/CreditTradeService.py b/backend/api/services/CreditTradeService.py index fd48cad19..d6c7713de 100644 --- a/backend/api/services/CreditTradeService.py +++ b/backend/api/services/CreditTradeService.py @@ -4,6 +4,7 @@ from collections import defaultdict, namedtuple from dateutil.relativedelta import relativedelta from django.utils import timezone +from decimal import Decimal from django.core.exceptions import ValidationError from django.db.models import Q @@ -190,6 +191,23 @@ def approve(credit_trade, update_user=None, batch_process=False): effective_date = credit_trade.trade_effective_date \ if credit_trade.trade_effective_date and credit_trade.trade_effective_date > today else today + # Check if the transaction is an administrative adjustment and + # if it would result in a negative balance for the organization + if credit_trade.type.the_type == "Administrative Adjustment": + org_balance = Decimal(credit_trade.respondent.organization_balance['validated_credits']) + + # Adjust org_balance to accont for pending deductions + if 'deductions' in credit_trade.respondent.organization_balance: + org_balance -= Decimal(credit_trade.respondent.organization_balance['deductions']) + + if org_balance <= 0 and credit_trade.number_of_credits < Decimal(0): + raise ValidationError(f"Organization {credit_trade.respondent.name} already has a balance of {org_balance}") + + # number_of_credits can be negative so we add to org balance here + if org_balance + credit_trade.number_of_credits < Decimal(0): + credit_trade.number_of_credits = -org_balance # Adjust the number of credits to prevent a negative balance + credit_trade.save() # Save the updated number_of_credits to the database + # Only transfer credits if the effective_date is today or in the past if effective_date <= today: CreditTradeService.transfer_credits( diff --git a/backend/api/services/OrganizationService.py b/backend/api/services/OrganizationService.py index 38fb823e3..c31be6e6e 100644 --- a/backend/api/services/OrganizationService.py +++ b/backend/api/services/OrganizationService.py @@ -1,5 +1,5 @@ import datetime -from django.db.models import Q, Sum, Count +from django.db.models import Q, Sum, Count, Case, When, F from api.models.ComplianceReport import ComplianceReport from api.models.CreditTrade import CreditTrade @@ -22,7 +22,6 @@ def get_pending_transfers_value(organization): Q(is_rescinded=False) & (Q(trade_effective_date__gte=datetime.datetime.now()) | Q(trade_effective_date__isnull=True))) ).aggregate(total_credits=Sum('number_of_credits')) - if pending_trades['total_credits'] is not None: pending_transfers_value = pending_trades['total_credits'] @@ -58,7 +57,6 @@ def get_pending_deductions( "Deleted" ]) ).filter(id=group_id).first() - if compliance_report and compliance_report.summary: if compliance_report.supplements_id and \ compliance_report.supplements_id > 0: @@ -107,12 +105,11 @@ def get_pending_deductions( elif compliance_report.status.director_status_id not in \ ["Rejected"]: if compliance_report.summary.credits_offset is not None: - deductions += compliance_report.summary.credits_offset + deductions += compliance_report.summary.credits_offset # if report.status.director_status_id == 'Accepted' and \ # ignore_pending_supplemental: # deductions -= report.summary.credits_offset - if deductions < 0: deductions = 0 @@ -126,7 +123,6 @@ def get_max_credit_offset(organization, compliance_year, exclude_reserved=False) compliance_period_effective_date = datetime.date( int(compliance_year), 1, 1 ) - credits = CreditTrade.objects.filter( (Q(status__status="Approved") & Q(type__the_type="Sell") & @@ -149,9 +145,14 @@ def get_max_credit_offset(organization, compliance_year, exclude_reserved=False) Q(status__status="Approved") & Q(respondent_id=organization.id) & Q(is_rescinded=False) & - Q(compliance_period__effective_date__lte=compliance_period_effective_date)) + Q(compliance_period__effective_date__lte=compliance_period_effective_date)) | + (Q(type__the_type="Administrative Adjustment") & + Q(status__status="Approved") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(number_of_credits__gte=0) & + Q(trade_effective_date__lte=effective_date_deadline)) ).aggregate(total=Sum('number_of_credits')) - debits = CreditTrade.objects.filter( (Q(status__status="Approved") & Q(type__the_type="Sell") & @@ -167,8 +168,25 @@ def get_max_credit_offset(organization, compliance_year, exclude_reserved=False) Q(status__status="Approved") & Q(respondent_id=organization.id) & Q(is_rescinded=False) & - Q(compliance_period__effective_date__lte=compliance_period_effective_date)) - ).aggregate(total=Sum('number_of_credits')) + Q(compliance_period__effective_date__lte=compliance_period_effective_date)) | + (Q(type__the_type="Administrative Adjustment") & + Q(status__status="Approved") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(number_of_credits__lt=0) & + Q(trade_effective_date__lte=effective_date_deadline)) + ).aggregate( + total=Sum( + Case( + When( + Q(type__the_type="Administrative Adjustment") & + Q(number_of_credits__lt=0), + then=F('number_of_credits') * -1 + ), + default=F('number_of_credits') + ) + ) + ) total_in_compliance_period = 0 if credits and credits.get('total') is not None: @@ -181,6 +199,99 @@ def get_max_credit_offset(organization, compliance_year, exclude_reserved=False) else: pending_deductions = OrganizationService.get_pending_deductions(organization, ignore_pending_supplemental=False) + validated_credits = organization.organization_balance.get( + 'validated_credits', 0 + ) + total_balance = validated_credits - pending_deductions + total_available_credits = min(total_in_compliance_period, total_balance) + if total_available_credits < 0: + total_available_credits = 0 + + return total_available_credits + + + @staticmethod + def get_max_credit_offset_for_interval(organization, compliance_date): + effective_date_deadline = compliance_date.date() + effective_year = effective_date_deadline.year + if effective_date_deadline < datetime.date(effective_year, 4, 1): + effective_year -= 1 + compliance_period_effective_date = datetime.date( + int(effective_year), 1, 1 + ) + + credits = CreditTrade.objects.filter( + (Q(status__status="Approved") & + Q(type__the_type="Sell") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(trade_effective_date__lte=effective_date_deadline)) | + (Q(status__status="Approved") & + Q(type__the_type="Buy") & + Q(initiator_id=organization.id) & + Q(is_rescinded=False) & + Q(trade_effective_date__lte=effective_date_deadline)) | + (Q(type__the_type="Part 3 Award") & + Q(status__status="Approved") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(trade_effective_date__lte=effective_date_deadline)) | + (Q(type__the_type="Credit Validation") & + Q(status__status="Approved") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(compliance_period__effective_date__lte=compliance_period_effective_date)) | + (Q(type__the_type="Administrative Adjustment") & + Q(status__status="Approved") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(number_of_credits__gte=0) & + Q(trade_effective_date__lte=effective_date_deadline)) + ).aggregate(total=Sum('number_of_credits')) + + debits = CreditTrade.objects.filter( + (Q(status__status="Approved") & + Q(type__the_type="Sell") & + Q(initiator_id=organization.id) & + Q(is_rescinded=False) & + Q(trade_effective_date__lte=effective_date_deadline)) | + (Q(status__status="Approved") & + Q(type__the_type="Buy") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(trade_effective_date__lte=effective_date_deadline)) | + (Q(type__the_type="Credit Reduction") & + Q(status__status="Approved") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(compliance_period__effective_date__lte=compliance_period_effective_date)) | + (Q(type__the_type="Administrative Adjustment") & + Q(status__status="Approved") & + Q(respondent_id=organization.id) & + Q(is_rescinded=False) & + Q(number_of_credits__lt=0) & + Q(trade_effective_date__lte=effective_date_deadline)) + ).aggregate( + total=Sum( + Case( + When( + Q(type__the_type="Administrative Adjustment") & + Q(number_of_credits__lt=0), + then=F('number_of_credits') * -1 + ), + default=F('number_of_credits') + ) + ) + ) + + total_in_compliance_period = 0 + if credits and credits.get('total') is not None: + total_in_compliance_period = credits.get('total') + + if debits and debits.get('total') is not None: + total_in_compliance_period -= debits.get('total') + pending_deductions = OrganizationService.get_pending_deductions(organization, ignore_pending_supplemental=False) + validated_credits = organization.organization_balance.get( 'validated_credits', 0 ) diff --git a/backend/api/services/SpreadSheetBuilder.py b/backend/api/services/SpreadSheetBuilder.py index 885771b2f..de09d806f 100644 --- a/backend/api/services/SpreadSheetBuilder.py +++ b/backend/api/services/SpreadSheetBuilder.py @@ -92,12 +92,12 @@ def add_credit_transfers(self, credit_trades, user): """ Adds a spreadsheet for credit transfers """ - worksheet = self.workbook.add_sheet("Credit Transactions") + worksheet = self.workbook.add_sheet("Transactions") row_index = 0 columns = [ - "Transaction ID", "Compliance Period", "Type", "Credits From", - "Credits To", "Quantity of Credits", "Value per Credit", "Category", + "Transaction ID", "Compliance Period", "Type", "Compliance Units From", + "Compliance Units To", "Number of Units", "Value per unit", "Category", "Status", "Effective Date", "Comments" ] @@ -151,7 +151,9 @@ def add_credit_transfers(self, credit_trades, user): worksheet.write(row_index, 8, credit_trade.status.friendly_name) # If the trade doesn't have an effective date but meets certain other criteria, write the update timestamp. - if credit_trade.update_timestamp: + if credit_trade.trade_effective_date: + worksheet.write(row_index, 9, credit_trade.trade_effective_date, date_format) + elif credit_trade.update_timestamp: # Conditions for using the update timestamp. approved_status = credit_trade.status.status == "Approved" valid_trade_type = credit_trade.type.the_type in ["Credit Reduction", "Credit Validation"] @@ -181,17 +183,18 @@ def add_credit_transfers(self, credit_trades, user): worksheet.col(9).width = 3500 worksheet.col(10).width = 10000 - def add_fuel_suppliers(self, fuel_suppliers): + def add_fuel_suppliers(self, fuel_suppliers, include_actions=False): """ Adds a spreadsheet for fuel suppliers """ - worksheet = self.workbook.add_sheet("Fuel Suppliers") + worksheet = self.workbook.add_sheet("Organizations") row_index = 0 columns = [ - "ID", "Organization Name", "Credit Balance", "Status", "Actions" + "ID", "Organization Name", "Compliance Units", "Registered" ] - + if include_actions: + columns.append("Actions") header_style = xlwt.easyxf('font: bold on') # Build Column Headers @@ -207,13 +210,18 @@ def add_fuel_suppliers(self, fuel_suppliers): worksheet.write( row_index, 2, fuel_supplier.organization_balance['validated_credits']) - worksheet.write(row_index, 3, fuel_supplier.status.status) - worksheet.write(row_index, 4, fuel_supplier.actions_type.the_type) + + # Adjust the value for the 'Registered' column based on the status + registered_status = 'Yes' if fuel_supplier.status.status.lower() == 'active' else 'No' + worksheet.write(row_index, 3, registered_status) + if include_actions: + worksheet.write(row_index, 4, fuel_supplier.actions_type.the_type) # set the widths for the columns that we expect to be longer worksheet.col(1).width = 7500 worksheet.col(2).width = 3500 - worksheet.col(4).width = 3500 + if include_actions: + worksheet.col(4).width = 3500 def add_users(self, fuel_supplier_users): """ diff --git a/backend/api/tests/base_test_case.py b/backend/api/tests/base_test_case.py index 2654ab6c3..e25ddc1f3 100644 --- a/backend/api/tests/base_test_case.py +++ b/backend/api/tests/base_test_case.py @@ -128,7 +128,10 @@ def setUp(self): self.credit_trade_types = { 'buy': CreditTradeType.objects.get(the_type='Buy'), 'sell': CreditTradeType.objects.get(the_type='Sell'), - 'part3award': CreditTradeType.objects.get(the_type='Part 3 Award') + 'part3award': CreditTradeType.objects.get(the_type='Part 3 Award'), + 'adminAdjustment': CreditTradeType.objects.get(the_type='Administrative Adjustment'), + 'creditReduction': CreditTradeType.objects.get(the_type='Credit Reduction'), + 'creditValidation': CreditTradeType.objects.get(the_type='Credit Validation'), } self.organizations = { diff --git a/backend/api/tests/payloads/compliance_unit_payloads.py b/backend/api/tests/payloads/compliance_unit_payloads.py new file mode 100644 index 000000000..27de03f1f --- /dev/null +++ b/backend/api/tests/payloads/compliance_unit_payloads.py @@ -0,0 +1,156 @@ +compliance_unit_initial_payload = { + 'status': { + 'fuelSupplierStatus': 'Draft' + }, + "scheduleB": { + "records": [ + { + "fuelCode": "21", + "fuelType": "Ethanol", + "fuelClass": "Gasoline", + "provisionOfTheAct": "Section 6 (5) (c)", + "quantity": 117933500 + }, + { + "fuelCode": None, + "fuelType": "Petroleum-based diesel", + "fuelClass": "Diesel", + "provisionOfTheAct": "Section 6 (5) (b)", + "quantity": 136896000 + } + ] + }, + 'summary': { + 'creditsOffset': 0, + } +} + +compliance_unit_supplemental_payload = { + 'status': { + 'fuelSupplierStatus': 'Submitted' + }, + "scheduleB": { + "records": [ + { + "fuelCode": "21", + "fuelType": "Ethanol", + "fuelClass": "Gasoline", + "provisionOfTheAct": "Section 6 (5) (c)", + "quantity": 117933500 + }, + { + "fuelCode": None, + "fuelType": "Petroleum-based diesel", + "fuelClass": "Diesel", + "provisionOfTheAct": "Section 6 (5) (b)", + "quantity": 136896000 + } + ] + }, + 'summary': { + 'creditsOffset': 0, + 'creditOffsetA': 0, + 'creditOffsetB': 0, + 'creditOffsetC': 0, + }, + 'supplemental_note': 'test compliance units' +} + +compliance_unit_positive_offset_payload = { + 'status': { + 'fuelSupplierStatus': 'Draft' + }, + "scheduleB": { + "records": [ + { + "fuelCode": "21", + "fuelType": "Ethanol", + "fuelClass": "Gasoline", + "provisionOfTheAct": "Section 6 (5) (c)", + "quantity": 117933500 + } + ] + }, + 'summary': { + 'creditsOffset': 0, + } +} + +compliance_unit_negative_offset_payload = { + 'status': { + 'fuelSupplierStatus': 'Draft' + }, + "scheduleB": { + "records": [ + { + "fuelCode": None, + "fuelType": "Petroleum-based diesel", + "fuelClass": "Diesel", + "provisionOfTheAct": "Section 6 (5) (b)", + "quantity": 136896000 + } + ] + }, + "summary": { + "dieselClassDeferred": 0, + "dieselClassObligation": 0, + "dieselClassPreviouslyRetained": 0, + "dieselClassRetained": 0, + "gasolineClassDeferred": 0, + "gasolineClassObligation": 0, + "gasolineClassPreviouslyRetained": 0, + "gasolineClassRetained": 0, + "creditsOffset": 0, + "creditsOffsetA": 0, + "creditsOffsetB": 0, + "creditsOffsetC": 0 + } +} + +compliance_unit_positive_supplemental_payload = { + 'status': { + 'fuelSupplierStatus': 'Submitted' + }, + "scheduleB": { + "records": [ + { + "fuelCode": "21", + "fuelType": "Ethanol", + "fuelClass": "Gasoline", + "provisionOfTheAct": "Section 6 (5) (c)", + "quantity": 117933500 + } + ] + }, + 'summary': { + 'creditsOffset': 0, + 'creditOffsetA': 0, + 'creditOffsetB': 0, + 'creditOffsetC': 0, + }, + 'supplemental_note': 'test compliance units' +} + +compliance_unit_negative_supplemental_payload = { + 'status': { + 'fuelSupplierStatus': 'Submitted' + }, + "scheduleB": { + "records": [ + { + "fuelCode": None, + "fuelType": "Petroleum-based diesel", + "fuelClass": "Diesel", + "provisionOfTheAct": "Section 6 (5) (b)", + "quantity": 136896000 + } + ] + }, + 'summary': { + 'creditsOffset': 0, + 'creditOffsetA': 0, + 'creditOffsetB': 0, + 'creditOffsetC': 0, + }, + 'supplemental_note': 'test compliance units' +} \ No newline at end of file diff --git a/backend/api/tests/test_compliance_supplemental_reporting.py b/backend/api/tests/test_compliance_supplemental_reporting.py index 1658657be..8a7c2811b 100644 --- a/backend/api/tests/test_compliance_supplemental_reporting.py +++ b/backend/api/tests/test_compliance_supplemental_reporting.py @@ -156,4 +156,4 @@ def test_supplemental_accepted_by_director_success(self): content_type='application/json', data=json.dumps(obj['payload']) ) - self.assertEqual(response.status_code, 200) \ No newline at end of file + self.assertEqual(response.status_code, 200) diff --git a/backend/api/tests/test_compliance_unit_reporting_after_2023.py b/backend/api/tests/test_compliance_unit_reporting_after_2023.py new file mode 100644 index 000000000..a179804bf --- /dev/null +++ b/backend/api/tests/test_compliance_unit_reporting_after_2023.py @@ -0,0 +1,1016 @@ +# -*- coding: utf-8 -*- +# pylint: disable=no-member,invalid-name +""" + REST API Documentation for the NRsS TFRS Credit Trading Application + The Transportation Fuels Reporting System is being designed to streamline + compliance reporting for transportation fuel suppliers in accordance with + the Renewable & Low Carbon Fuel Requirements Regulation. + OpenAPI spec version: v1 + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +""" +import json +import logging +from decimal import Decimal + +from django.utils import timezone +from rest_framework import status + +from api.models.CompliancePeriod import CompliancePeriod +from api.models.ComplianceReport import ComplianceReport, ComplianceReportStatus, ComplianceReportType, \ + ComplianceReportWorkflowState +from api.models.Organization import Organization +from .base_test_case import BaseTestCase +from .payloads.compliance_unit_payloads import * +from ..services.OrganizationService import OrganizationService + +logger = logging.getLogger('supplemental_reporting') +logger.setLevel(logging.INFO) +handler = logging.StreamHandler() +formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') +handler.setFormatter(formatter) +logger.addHandler(handler) +COMPLIANCE_YEAR = '2023' + +class TestComplianceUnitReporting(BaseTestCase): + """Tests for the compliance unit reporting and supplemental reporting endpoints""" + extra_fixtures = [ + 'test/test_post_compliance_unit_reporting.json', + 'test/test_fuel_codes.json', + 'test/test_unit_of_measures.json', + 'test/test_carbon_intensity_limits.json', + 'test/test_default_carbon_intensities.json', + 'test/test_petroleum_carbon_intensities.json', + 'test/test_transaction_types.json' + ] + + def _create_draft_compliance_report(self, report_type="Compliance Report"): + report = ComplianceReport() + report.status = ComplianceReportWorkflowState.objects.create( + fuel_supplier_status=ComplianceReportStatus.objects.get_by_natural_key('Draft') + ) + report.organization = Organization.objects.get_by_natural_key( + "Test Org 1") + report.compliance_period = CompliancePeriod.objects.get_by_natural_key(COMPLIANCE_YEAR) + report.type = ComplianceReportType.objects.get_by_natural_key(report_type) + report.create_timestamp = timezone.now() + report.update_timestamp = timezone.now() + + report.save() + report.refresh_from_db() + return report.id + + def _add_or_remove_credits(self, num_of_credits, validation=True): + # Create a recommended credit trade request i.e., either reduction or validation using historical data entry. + payload = { + "compliancePeriod": CompliancePeriod.objects.get_by_natural_key(COMPLIANCE_YEAR).id, + "initiator": self.users['gov_director'].organization.id, + "numberOfCredits": num_of_credits, + "respondent": self.users['fs_user_1'].organization.id, + "status": self.statuses['recommended'].id, + "tradeEffectiveDate": "2021-01-01", + "type": self.credit_trade_types['creditValidation'].id if validation else self.credit_trade_types['creditReduction'].id, + "is_rescinded": False, + "zeroReason": None, + "comment": "testing" + } + response = self.clients['gov_multi_role'].post( + '/api/credit_trades', + content_type='application/json', + data=json.dumps(payload)) + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + ct_id = response.data['id'] + # Approve the credit trade + payload['status'] = self.statuses['approved'].id + response = self.clients['gov_multi_role'].put( + '/api/credit_trades/{}'.format(ct_id), + content_type='application/json', + data=json.dumps(payload)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + def _create_supplemental_report(self, rid): + payload = { + 'supplements': rid, + 'status': {'fuelSupplierStatus': 'Draft'}, + 'type': 'Compliance Report', + 'compliancePeriod': COMPLIANCE_YEAR + } + response = self.clients['fs_user_1'].post( + '/api/compliance_reports', + content_type='application/json', + data=json.dumps(payload) + ) + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + sid = response.data['id'] + return sid + + def _patch_fs_user_for_compliance_report(self, payload, cr_id): + response = self.clients['fs_user_1'].patch( + '/api/compliance_reports/{id}'.format(id=cr_id), + content_type='application/json', + data=json.dumps(payload) + ) + self.assertEqual(response.status_code, status.HTTP_200_OK) + return response + + def _acceptance_from_director(self, cr_id): + # we are only allowed to change one status at a time so this + # loops the statuses in order to get to accepted by director + status_payloads = [ + {'user': 'gov_analyst', 'payload': {'status': {'analystStatus': 'Recommended'}}}, + {'user': 'gov_manager', 'payload': {'status': {'managerStatus': 'Recommended'}}}, + {'user': 'gov_director', 'payload': {'status': {'directorStatus': 'Accepted'}}} + ] + for obj in status_payloads: + response = self.clients[obj['user']].patch( + '/api/compliance_reports/{id}'.format(id=cr_id), + content_type='application/json', + data=json.dumps(obj['payload']) + ) + self.assertEqual(response.status_code, 200) + + """ + | Scenario 1: Initial Report - positive net balance | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Values | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + | Net compliance unit balance for compliance period | Line 25 | Z | 100 | + | Available compliance unit balance on March 31, YYYY | | A | 800 | + | Compliance unit balance change from assessment | | X | 100 | + | If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 900 | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + """ + def test_initial_report_positive_net_balance(self): + self._add_or_remove_credits(800) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_positive_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = {'status': {'fuelSupplierStatus': 'Submitted'}} + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data.get('summary').get('lines').get('25'), 100) + self.assertEqual(response.data.get('summary').get('lines').get('29A'), 800) + self.assertEqual(response.data.get('summary').get('lines').get('29B'), 100) + self.assertEqual(response.data.get('summary').get('lines').get('28'), 0) + self.assertEqual(response.data.get('summary').get('lines').get('29C'), 900) + + """ + | Scenario 2: Initial Report - negative net balance, no penalty | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Values | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -200 | + | Available compliance unit balance on March 31, YYYY | | A | 700 | + | Compliance unit balance change from assessment | | X | -200 | + | If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 500 | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + """ + def test_initial_report_negative_net_balance_no_penalty(self): + self._add_or_remove_credits(700) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 342238 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -200 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -200) + self.assertEqual(response.data.get('summary').get('lines').get('29A'), 700) + self.assertEqual(response.data.get('summary').get('lines').get('29B'), -200) + self.assertEqual(response.data.get('summary').get('lines').get('28'), 0) + self.assertEqual(response.data.get('summary').get('lines').get('29C'), 500) + + """ + | Scenario 3: Initial Report - negative net balance with penalty | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Values | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -400 | + | Available compliance unit balance on March 31, YYYY | | A | 300 | + | Compliance unit balance change from assessment | | X | -300 | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | + | Non-compliance penalty payable (100 units * $600 CAD per unit) | Line 28 | | 60,000 | + | ^---- (abs(Z) - A) * $600 | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 0 | + |-----------------------------------------------------------------------|---------------|-------------|-----------------| + """ + def test_initial_report_negative_net_balance_with_penalty(self): + self._add_or_remove_credits(300) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 684477 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -300 + self._patch_fs_user_for_compliance_report(payload, rid) + + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -400) + self.assertEqual(response.data.get('summary').get('lines').get('29A'), 300) + self.assertEqual(response.data.get('summary').get('lines').get('29B'), -300) + self.assertEqual(response.data.get('summary').get('lines').get('28'), 60000) + self.assertEqual(response.data.get('summary').get('lines').get('29C'), 0) + + def test_initial_report_zero_starting_value(self): + self._add_or_remove_credits(0) # Starting with zero credits + rid = self._create_draft_compliance_report() + + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 684477 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 0 + self._patch_fs_user_for_compliance_report(payload, rid) + + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + self.assertEqual(response.data.get('summary').get('lines').get('25'), -400) + self.assertEqual(response.data.get('summary').get('lines').get('29A'), 0) + self.assertEqual(response.data.get('summary').get('lines').get('29B'), 0) + self.assertEqual(response.data.get('summary').get('lines').get('28'), 240000) + self.assertEqual(response.data.get('summary').get('lines').get('29C'), 0) + + """ + | Scenario 4: Supplemental Report Submission #1, increasing, positive net balance, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | 100 | 250 | 150 | + | Available compliance unit balance on March 31, YYYY | | A | 800 | 900 | 100 | + | Compliance unit balance change from assessment | | X | 100 | 150 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 900 | 1,050 | 150 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_increasing_positive_net_balance_previous_report_assessed(self): + self._add_or_remove_credits(800) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_positive_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(800)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(900)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_positive_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 294833 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, sid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the supplemental compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(250)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(900)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(150)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(1050)) + + """ + | Scenario 5: Supplemental Report Submission #4, decreasing, positive net balance, no penalty, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | 100 | 50 | -50 | + | Available compliance unit balance on March 31, YYYY | | A | 800 | 900 | 100 | + | Compliance unit balance change from assessment | | X | 100 | -50 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 900 | 850 | -50 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_4_decreasing_positive_net_balance_previous_report_assessed(self): + self._add_or_remove_credits(800) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_positive_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(800)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(900)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_positive_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 58967 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(50)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(900)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-50)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(850)) + + """ + | Scenario 6: Supplemental Report Submission #1, decreasing, positive net balance, with penalty, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | 100 | 50 | -50 | + | Available compliance unit balance on March 31, YYYY | | A | 0 | 25 | 25 | + | Compliance unit balance change from assessment | | X | 100 | -25 | -125 | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Non-compliance penalty payable (25 units * $600 CAD per unit) | Line 28 | | | $15,000 | $15,000 | + | ^---- (abs(Z) - A) * $600 | | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 100 | 0 | -100 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | * Scenario 6 could occur if the organization sold 75 compliance units after the initial report was assessed and then had to submit a supplemental when they only had 25 compliance units remaining + """ + def test_supplemental_report_1_decreasing_positive_net_balance_penalty_previous_report_assessed(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_positive_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(100)) + # remove 75 units from Org balance to create the scenario of it selling them + self._add_or_remove_credits(75, False) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_positive_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 58967 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(50)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(25)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-25)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(15000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + + """ + | Scenario 7: Supplemental Report Submission #1, increasing, negative net balance, no penalty, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -200 | -100 | 100 | + | Available compliance unit balance on March 31, YYYY | | A | 800 | 600 | -200 | + | Compliance unit balance change from assessment | | X | -200 | 100 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 600 | 700 | 100 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_increasing_negative_net_balance_no_penalty_previous_report_assessed(self): + self._add_or_remove_credits(800) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 342238 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -200 + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(800)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(600)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_negative_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 171119 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -100 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(600)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(700)) + + """ + | Scenario 8: Supplemental Report Submission #1, increasing, negative net balance, with penalty, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -400 | -350 | 50 | + | Available compliance unit balance on March 31, YYYY | | A | 100 | 0 | -100 | + | Compliance unit balance change from assessment | | X | -100 | 50 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Non-compliance penalty payable (250 units * $600 CAD per unit) | Line 28 | | $180,000 | $150,000 | -$30,000 | + | ^---- (abs(Z) - A) * $600 | | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 0 | 0 | 0 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_increasing_negative_net_balance_penalty_previous_report_assessed(self): + self._add_or_remove_credits(100) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 684477 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -400 + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(180000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_negative_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 598917 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -350 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-350)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(50)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(150000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + + """ + | Scenario 9: Supplemental Report Submission #1, decreasing, negative net balance, no penalty, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -200 | -300 | -100 | + | Available compliance unit balance on March 31, YYYY | | A | 800 | 600 | -200 | + | Compliance unit balance change from assessment | | X | -200 | -100 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 600 | 500 | -100 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_decreasing_negative_net_balance_no_penalty_previous_report_assessed(self): + self._add_or_remove_credits(800) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 342238 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -200 + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(800)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(600)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_negative_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 513358 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -300 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-300)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(600)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(500)) + + """ + | Scenario 10: Supplemental Report Submission #1, decreasing, negative net balance, with penalty, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -200 | -300 | -100 | + | Available compliance unit balance on March 31, YYYY | | A | 800 | 0 | -800 | + | Compliance unit balance change from assessment | | X | -200 | -100 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Non-compliance penalty payable (250 units * $600 CAD per unit) | Line 28 | | | $60,000 | $60,000 | + | ^---- (abs(Z) - A) * $600 | | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 600 | 0 | -600 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_decreasing_negative_net_balance_penalty_previous_report_assessed(self): + self._add_or_remove_credits(800) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 342238 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -200 + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(800)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(600)) + # * Scenario 10 could occur if the organization sold all 600 compliance units after the initial report was + # assessed and then had to submit a supplemental when they had 0 compliance units in their available balance + self._add_or_remove_credits(600, False) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_negative_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 513358 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -300 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-300)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(60000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + + """ + | Scenario 11: Supplemental Report Submission #1, increasing, negative net balance to positive net balance, no penalty, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -300 | 200 | 500 | + | Available compliance unit balance on March 31, YYYY | | A | 500 | 200 | -300 | + | Compliance unit balance change from assessment | | X | -300 | 500 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 200 | 700 | 500 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_increasing_negative_net_balance_to_positive_no_penalty_previous_report_assessed(self): + self._add_or_remove_credits(500) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 513358 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -300 + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-300)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(500)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-300)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(200)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_positive_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 235867 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(500)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(700)) + + """ + | Scenario 12: Supplemental Report Submission #1, decreasing, positive net balance to negative net balance, with penalty, previous report was assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | 200 | -100 | -300 | + | Available compliance unit balance on March 31, YYYY | | A | 0 | 200 | 200 | + | Compliance unit balance change from assessment | | X | 200 | -200 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Non-compliance penalty payable (250 units * $600 CAD per unit) | Line 28 | | | $60,000 | $60,000 | + | ^---- (abs(Z) - A) * $600 | | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 200 | 0 | -200 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_decreasing_positive_net_balance_to_negative_penalty_previous_report_assessed(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_positive_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 235867 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(200)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_negative_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 171119 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -100 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(60000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + + """ + | Scenario 13: Supplemental Report Submission #1, increasing, positive net balance, previous report was not assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | 300 | 400 | 100 | + | Available compliance unit balance on March 31, YYYY | | A | 1000 | 1000 | 0 | + | Compliance unit balance change from assessment | | X | 300 | 400 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 1300 | 1400 | 100 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_increasing_positive_net_balance_previous_report_not_assessed(self): + self._add_or_remove_credits(1000) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_positive_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 353800 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report which is not accessed and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(300)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(1000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(300)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(1300)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_positive_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 471733 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(1000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(1400)) + + """ + | Scenario 14: Supplemental Report Submission #1, increasing, negative net balance, no penalty, previous report was not assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -200 | -300 | -100 | + | Available compliance unit balance on March 31, YYYY | | A | 1000 | 1000 | 0 | + | Compliance unit balance change from assessment | | X | -200 | -300 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 800 | 700 | -100 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_increasing_negative_net_balance_no_penalty_previous_report_not_assessed(self): + self._add_or_remove_credits(1000) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 342238 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -200 + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report which is not accessed and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(1000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(800)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_negative_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 513358 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -300 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-300)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(1000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-300)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(700)) + + """ + | Scenario 15: Supplemental Report Submission #1, decreasing, negative net balance, with penalty, previous report was not assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -400 | -600 | -200 | + | Available compliance unit balance on March 31, YYYY | | A | 200 | 200 | 0 | + | Compliance unit balance change from assessment | | X | -200 | -200 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Non-compliance penalty payable (400 units * $600 CAD per unit) | Line 28 | | $120,000 | $240,000 | $120,000 | + | ^---- (abs(Z) - A) * $600 | | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 0 | 0 | 0 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_decreasing_negative_net_balance_penalty_previous_report_not_assessed(self): + self._add_or_remove_credits(200) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 684477 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -400 + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report which is not accessed and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(120000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_negative_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 1026715 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -600 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-600)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(240000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + + """ + | Scenario 16: Supplemental Report Submission #1, no change to net balance, positive net balance, previous report was not assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | 400 | 400 | 0 | + | Available compliance unit balance on March 31, YYYY | | A | 1000 | 1000 | 0 | + | Compliance unit balance change from assessment | | X | 400 | 400 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 1400 | 0 | 0 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_no_change_positive_net_balance_previous_report_not_assessed(self): + self._add_or_remove_credits(1000) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_positive_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 471733 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report which is not accessed and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(1000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(1400)) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_positive_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 471733 # credits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(1000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(0)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(1400)) + + """ + | Scenario 17: Supplemental Report Submission #1, no change to net balance, negative net balance, penalty, previous report was not assessed | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Low Carbon Fuel Requirement Summary | |Calculations | Example Old Values | Example New Values | Example Change Values | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + | Net compliance unit balance for compliance period | Line 25 | Z | -400 | -400 | 0 | + | Available compliance unit balance on March 31, YYYY | | A | 200 | 100 | -100 | + | Compliance unit balance change from assessment | | X | -200 | -100 | | + | ^---- If Z>0, then Z; If Z<0 & A+Z>0, then Z; If Z<0 & A+Z<0, then -A| | | | + | Non-compliance penalty payable (400 units * $600 CAD per unit) | Line 28 | | $120,000 | $180,000 | $60,000 | + | ^---- (abs(Z) - A) * $600 | | | | + | Available compliance unit balance after assessment on March 31, YYYY | | A+X | 0 | 0 | 0 | + |-----------------------------------------------------------------------|---------------|-------------|---------------------|--------------------|-----------------------| + """ + def test_supplemental_report_1_no_change_negative_net_balance_penalty_previous_report_not_assessed(self): + self._add_or_remove_credits(200) + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 684477 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -400 + self._patch_fs_user_for_compliance_report(payload, rid) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report which is not accessed and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-200)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(120000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + # * Scenario 17 could occur if the organization submitted a supplemental report for a previous compliance period + # after they submitted the initial report for this period; processing the supplemental report from the previous + # period could lead to a decrease in the available credit balance for this compliance period + self._add_or_remove_credits(100, False) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_negative_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 684477 # debits from fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = -400 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-400)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-100)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(180000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + + """Testing that even in penalty situation, organization balances never go below zero""" + def test_organization_balance_never_below_zero(self): + self._add_or_remove_credits(100000) + + organization = Organization.objects.get_by_natural_key("Test Org 1") + initial_balance = organization.organization_balance + + self.assertEqual(initial_balance['validated_credits'], Decimal(100000)) + + # Create inital draft report + rid = self._create_draft_compliance_report() + + # Patch compliance report info + payload = compliance_unit_negative_offset_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 342238999 # debits from fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + } + self._patch_fs_user_for_compliance_report(payload, rid) + + # Successful director acceptance + self._acceptance_from_director(rid) + + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}/snapshot'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('25')), Decimal(-200000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29A')), Decimal(100000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29B')), Decimal(-100000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('28')), Decimal(60000000)) + self.assertEqual(Decimal(response.data.get('summary').get('lines').get('29C')), Decimal(0)) + + # Ensure that the organization balance is zero and not negative + updated_balance = organization.organization_balance + self.assertEqual(updated_balance['validated_credits'], 0) + + lastest_transaction = CreditTrade.objects.last() + # Optionally: Ensure that the credit transaction was made for the amount of available balance + # Replace with the actual way you get the transaction amount. + self.assertEqual(lastest_transaction.number_of_credits, initial_balance['validated_credits']) + diff --git a/backend/api/tests/test_compliance_unit_reporting_before_2023.py b/backend/api/tests/test_compliance_unit_reporting_before_2023.py new file mode 100644 index 000000000..7056a9dad --- /dev/null +++ b/backend/api/tests/test_compliance_unit_reporting_before_2023.py @@ -0,0 +1,849 @@ +# -*- coding: utf-8 -*- +# pylint: disable=no-member,invalid-name +""" + REST API Documentation for the NRsS TFRS Credit Trading Application + The Transportation Fuels Reporting System is being designed to streamline + compliance reporting for transportation fuel suppliers in accordance with + the Renewable & Low Carbon Fuel Requirements Regulation. + OpenAPI spec version: v1 + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +""" +import json +import logging + +from django.utils import timezone +from rest_framework import status + +from api.models.CompliancePeriod import CompliancePeriod +from api.models.ComplianceReport import ComplianceReport, ComplianceReportStatus, ComplianceReportType, \ + ComplianceReportWorkflowState +from api.models.Organization import Organization +from .base_test_case import BaseTestCase +from .payloads.compliance_unit_payloads import * +from ..services.OrganizationService import OrganizationService + +logger = logging.getLogger('supplemental_reporting') +logger.setLevel(logging.INFO) +handler = logging.StreamHandler() +formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') +handler.setFormatter(formatter) +logger.addHandler(handler) + + +class TestComplianceUnitReporting(BaseTestCase): + """Tests for the compliance unit reporting and supplemental reporting endpoints""" + extra_fixtures = [ + 'test/test_pre_compliance_unit_reporting.json', + 'test/test_fuel_codes.json', + 'test/test_unit_of_measures.json', + 'test/test_carbon_intensity_limits.json', + 'test/test_default_carbon_intensities.json', + 'test/test_petroleum_carbon_intensities.json', + 'test/test_transaction_types.json' + ] + + def _create_draft_compliance_report(self, report_type="Compliance Report"): + report = ComplianceReport() + report.status = ComplianceReportWorkflowState.objects.create( + fuel_supplier_status=ComplianceReportStatus.objects.get_by_natural_key('Draft') + ) + report.organization = Organization.objects.get_by_natural_key( + "Test Org 1") + report.compliance_period = CompliancePeriod.objects.get_by_natural_key('2022') + report.type = ComplianceReportType.objects.get_by_natural_key(report_type) + report.create_timestamp = timezone.now() + report.update_timestamp = timezone.now() + + report.save() + report.refresh_from_db() + return report.id + + def _add_part3_awards_to_org(self, add_credits): + # Create a recommended credit trade request + payload = { + "compliancePeriod": CompliancePeriod.objects.get_by_natural_key('2022').id, + "initiator": self.users['gov_director'].organization.id, + "numberOfCredits": add_credits, + "respondent": self.users['fs_user_1'].organization.id, + "status": self.statuses['recommended'].id, + "tradeEffectiveDate": "2021-01-01", + "type": self.credit_trade_types['part3award'].id, + "is_rescinded": False, + "zeroReason": None, + "comment": "testing" + } + response = self.clients['gov_multi_role'].post( + '/api/credit_trades', + content_type='application/json', + data=json.dumps(payload)) + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + ct_id = response.data['id'] + # Approve the credit trade + payload['status'] = self.statuses['approved'].id + response = self.clients['gov_multi_role'].put( + '/api/credit_trades/{}'.format(ct_id), + content_type='application/json', + data=json.dumps(payload)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + + def _create_supplemental_report(self, rid): + payload = { + 'supplements': rid, + 'status': {'fuelSupplierStatus': 'Draft'}, + 'type': 'Compliance Report', + 'compliancePeriod': '2022' + } + response = self.clients['fs_user_1'].post( + '/api/compliance_reports', + content_type='application/json', + data=json.dumps(payload) + ) + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + sid = response.data['id'] + return sid + + def _patch_fs_user_for_compliance_report(self, payload, cr_id): + response = self.clients['fs_user_1'].patch( + '/api/compliance_reports/{id}'.format(id=cr_id), + content_type='application/json', + data=json.dumps(payload) + ) + self.assertEqual(response.status_code, status.HTTP_200_OK) + return response + + def _acceptance_from_director(self, cr_id): + # we are only allowed to change one status at a time so this + # loops the statuses in order to get to accepted by director + status_payloads = [ + {'user': 'gov_analyst', 'payload': {'status': {'analystStatus': 'Recommended'}}}, + {'user': 'gov_manager', 'payload': {'status': {'managerStatus': 'Recommended'}}}, + {'user': 'gov_director', 'payload': {'status': {'directorStatus': 'Accepted'}}} + ] + for obj in status_payloads: + response = self.clients[obj['user']].patch( + '/api/compliance_reports/{id}'.format(id=cr_id), + content_type='application/json', + data=json.dumps(obj['payload']) + ) + self.assertEqual(response.status_code, 200) + + """ + | Scenario 0: Initial Report in a net credit position | + |--------------------------------------------------------------------------------------------------| + | Line | Description | Units | Example Value | + |------|-------------------------------------------------------------|-------------|---------------| + | 23 | Total credits from fuel supplied (from Schedule B) | Credits | 100,000 | + | 24 | Total debits from fuel supplied (from Schedule B) | (Debits) | 80,000 | + | 25 | Net credit or debit balance for the compliance period | Credits | 20,000 | + | 26 | Total banked credits used to offset outstanding debits | Credits | | + | 27 | Outstanding debit balance | (Debits) | | + | 28 | Part 3 non-compliance penalty payable | $CAD | | + |------|-------------------------------------------------------------|-------------|---------------| + | | Corresponding Compliance Unit conversion / transaction | | +20,000 | + """ + def test_initial_report_in_net_credit_position(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload.copy() + payload['status']['fuelSupplierStatus'] = 'Submitted' + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 20000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), 20000.0) + + """ + | Scenario 1: Initial Report in a net debit position | + |--------------------------------------------------------------------------------------------------| + | Line | Description | Units | Example Value | + |------|-------------------------------------------------------------|-------------|---------------| + | 23 | Total credits from fuel supplied (from Schedule B) | Credits | 105,000 | + | 24 | Total debits from fuel supplied (from Schedule B) | (Debits) | 150,000 | + | 25 | Net credit or debit balance for the compliance period | (Debits) | -45,000 | + | 26 | Total banked credits used to offset outstanding debits | Credits | 45,000 | + | 27 | Outstanding debit balance | (Debits) | 0 | + | 28 | Part 3 non-compliance penalty payable | $CAD | | + |------|-------------------------------------------------------------|-------------|---------------| + | | Corresponding Compliance Unit conversion / transaction | | -45,000 | + """ + def test_initial_report_in_net_debit_position(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 123830000 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 256679000 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(50000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 45000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check (50,000 - 45,000) [50,000 from org balance) + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 5000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -45000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 45000.0) + + """ + | Scenario 2: Supplemental Report Submission #1 that increases debit obligation | + |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | | Units | Example Values - Initial Submission | Example Values - Supplemental #1 | + |---------------------------------------------------------------------------|---------|--------------|-------------------------------------|-------------------------------------|----------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | Credits | 100,000 | 100,000 | + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | (Debits) | 145,000 | 150,000 | + | Net credit or debit balance for compliance period | Line 25 | Z | Credits (Debits) | -45,000 | -50,000 | + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A+B | Credits | 45,000 | 50,000 | + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a| A | Credits | n/a | 45,000 | + | Banked credits used to offset outstanding debits - Supplemental Report #1 | Line 26b| B (editable) | Credits | n/a | 5,000 | + | Outstanding debit balance | Line 27 | Z - (A+B) | (Debits) | 0 | 0 | + | Part 3 non-compliance penalty payable | Line 28 | | $CAD | | | + |---------------------------------------------------------------------------|---------|--------------|-------------------------------------|-------------------------------------|----------------------------------| + | Report Status (for compliance units conversion) | | | | Accepted | Accepted | + | Corresponding Compliance Unit conversion / transaction | | | | -45,000 | -5,000 | + """ + def test_supplemental_report_submission_increasing_debit_obligation(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 248122823 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(50000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 45000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check (50,000 - 45,000) [50,000 from org balance) + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 5000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -45000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 45000.0) + # Create supplemental report #1 + sid = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 256678782 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 50000 + payload['summary']['creditsOffsetA'] = 45000 + payload['summary']['creditsOffsetB'] = 5000 + self._patch_fs_user_for_compliance_report(payload, sid) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check (5000 - 5000) [5000 from org balance) + self.assertEqual( + min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 0) + self.assertEqual(response.data['summary']['lines']['25'], -50000.0) + self.assertEqual(response.data['summary']['lines']['26'], 50000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 45000.0) + self.assertEqual(response.data['summary']['lines']['26B'], 5000.0) + + """ + | Scenario 3: Supplemental Report Submission #2 that increases debit obligation - Example 1 | + |--------------------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | | Units | Initial Submission - Accepted | Example 1 - Supplemental #1 - Accepted | Example 1 - Supplemental #2 - Accepted | + |---------------------------------------------------------------------------|----------|---------------|-------------------------------|----------------------------------------|----------------------------------------|----------------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | Credits | 100,000 | 100,000 | 100,000 | + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | (Debits) | 145,000 | 148,000 | 150,000 | + | Net credit or debit balance for compliance period | Line 25 | Z | Credits (Debits) | -45,000 | -48,000 | -50,000 | + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A+B+C+D | Credits | 45,000 | 48,000 | 50,000 | + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a | A+B+C | Credits | n/a | 45,000 | 48,000 | + | Banked credits used to offset outstanding debits - Supplemental Report #2 | Line 26b | D (editable) | Credits | n/a | 3,000 | 2,000 | + | Outstanding debit balance | Line 27 | Z - (A+B+C+D) | (Debits) | 0 | 0 | 0 | + | Part 3 non-compliance penalty payable | Line 28 | | $CAD | $- | $- | $- | + | Report Status (for compliance units conversion) | | | | Accepted | Accepted | Accepted | + | Corresponding Compliance Unit conversion / transaction | | | | -45,000 | -3,000 | -2,000 | + """ + def test_supplemental_report_submission_2_ex1_increasing_debit_obligation(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 248122823 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(50000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 45000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check (50,000 - 45,000) [50,000 from org balance] + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 5000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -45000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 45000.0) + # Create supplemental report #1 + sid1 = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 253256398 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 48000 + payload['summary']['creditsOffsetA'] = 45000 + payload['summary']['creditsOffsetB'] = 3000 + self._patch_fs_user_for_compliance_report(payload, sid1) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid1) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid1)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], -48000.0) + self.assertEqual(response.data['summary']['lines']['26'], 48000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 45000.0) + self.assertEqual(response.data['summary']['lines']['26B'], 3000.0) + # compliance unit balance check (5000 - 3000) [5000 from org balance] + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 2000) + # Create supplemental report #2 + sid2 = self._create_supplemental_report(sid1) + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 256678782 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 50000 + payload['summary']['creditsOffsetA'] = 48000 + payload['summary']['creditsOffsetB'] = 2000 + self._patch_fs_user_for_compliance_report(payload, sid2) + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid2)) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid2) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid2)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], -50000.0) + self.assertEqual(response.data['summary']['lines']['26'], 50000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 48000.0) + self.assertEqual(response.data['summary']['lines']['26B'], 2000.0) + # compliance unit balance check (2000 - 2000) [2000 from org balance] + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 0) + + """ + | Scenario 3: Supplemental Report Submission #2 that increases debit obligation - Example 2 | + |--------------------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | | Units | Initial Submission - Accepted | Example 2 - Supplemental #1 - Submitted | Example 2 - Supplemental #2 - Accepted | + |---------------------------------------------------------------------------|----------|---------------|-------------------------------|--------------------------------|-----------------------------------------|----------------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | Credits | 100,000 | 100,000 | 100,000 + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | (Debits) | 144,000 | 148,000 | 150,000 + | Net credit or debit balance for compliance period | Line 25 | Z | Credits (Debits) | -44,000 | -48,000 | -50,000 + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A+B+C+D | Credits | 44,000 | 46,000 | 46,000 + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a | A+B+C | Credits | n/a | 44,000 | 44,000 + | Banked credits used to offset outstanding debits - Supplemental Report #2 | Line 26b | D (editable) | Credits | n/a | 2,000 | 2,000 + | Outstanding debit balance | Line 27 | Z - (A+B+C+D) | (Debits) | 0 | 2,000 | 4,000 + | Part 3 non-compliance penalty payable | Line 28 | | $CAD | $- | $400,000 | $800,000 + |---------------------------------------------------------------------------|----------|---------------|-------------------------------|--------------------------------|-----------------------------------------|----------------------------------------| + | Report Status (for compliance units conversion) | | | | Accepted | Submitted (not Accepted) | Accepted + |---------------------------------------------------------------------------|----------|---------------|-------------------------------|--------------------------------|-----------------------------------------|----------------------------------------| + | Corresponding Compliance Unit conversion / transaction | | | | -44,000 | | -2,000 + """ + def test_supplemental_report_submission_2_ex2_increasing_debit_obligation(self): + rid = self._create_draft_compliance_report() + # patch compliance report information + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 246411631 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(50000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 44000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -44000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 44000.0) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 6000) + # Create supplemental report #1 + sid1 = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 253256398 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 46000 + payload['summary']['creditsOffsetA'] = 44000 + payload['summary']['creditsOffsetB'] = 2000 + self._patch_fs_user_for_compliance_report(payload, sid1) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid1)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], -48000.0) + self.assertEqual(response.data['summary']['lines']['26'], 46000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 44000.0) + self.assertEqual(response.data['summary']['lines']['26B'], 2000.0) + self.assertEqual(response.data['summary']['lines']['27'], -2000.0) + self.assertEqual(response.data['summary']['lines']['28'], 400000.0) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 6000) + # Create supplemental report #2 + sid2 = self._create_supplemental_report(sid1) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 256678782 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 46000 + payload['summary']['creditsOffsetA'] = 44000 + payload['summary']['creditsOffsetB'] = 2000 + self._patch_fs_user_for_compliance_report(payload, sid2) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid2) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid2)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], -50000.0) + self.assertEqual(response.data['summary']['lines']['26'], 46000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 44000.0) + self.assertEqual(response.data['summary']['lines']['26B'], 2000.0) + self.assertEqual(response.data['summary']['lines']['27'], -4000.0) + self.assertEqual(response.data['summary']['lines']['28'], 800000.0) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 4000) + + """ + | Scenario 4: Supplemental Report Submission #1 that decreases debit obligation and still in a net debit position overall (Line 25) | + |-----------------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | | Units | Initial Submission - Accepted | Example 1 - Supplemental #1 - Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|-----------------|-------------------------------|----------------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | Credits | 100,000 | 100,000 | + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | (Debits) | 160,000 | 150,000 | + | Net credit or debit balance for compliance period | Line 25 | Z | Credits (Debits)| -60,000 | -50,000 | + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A-R | Credits | 60,000 | 50,000 | + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a | A | Credits | n/a | 60,000 | + | Banked credits used to offset outstanding debits - Supplemental Report #1 | Line 26b | Not editable | Credits | n/a | n/a | + | Outstanding debit balance | Line 27 | | (Debits) | 0 | 0 | + | Part 3 non-compliance penalty payable | Line 28 | | $CAD | | | + |-------------------------------------------------------------------------------------|----------|--------------|-----------------|-------------------------------|----------------------------------------| + | Banked credits to be returned as a result of supplemental reporting (if applicable) | | R | | | 10,000 | + |-------------------------------------------------------------------------------------|----------|--------------|-----------------|-------------------------------|----------------------------------------| + | Report Status (for compliance units conversion) | | | | Accepted | Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|-----------------|-------------------------------|----------------------------------------| + | Corresponding Compliance Unit conversion / transaction | | | | -60,000 | +10,000 | + """ + def test_supplemental_report_submission_1_ex1_decreasing_debit_obligation_under_net_debit_position(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 273790701 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(100000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 60000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 40000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -60000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 60000.0) + # Create supplemental report #1 + sid1 = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 256678782 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 50000 + payload['summary']['creditsOffsetA'] = 60000 + payload['summary']['creditsOffsetB'] = 0 + self._patch_fs_user_for_compliance_report(payload, sid1) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid1) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid1)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], -50000.0) + self.assertEqual(response.data['summary']['lines']['26'], 50000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 60000.0) + self.assertEqual(response.data['summary']['lines']['27'], 0) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 50000) + + """ + | Scenario 4: Supplemental Report Submission #1 that decreases debit obligation and still in a net debit position overall (Line 25) | + |-----------------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | | Units | Example 2 - Initial Submission - Accepted | Example 2 - Supplemental #1 - Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|-----------------|-------------------------------------------|----------------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | Credits | 100,000 | 100,000 + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | (Debits) | 160,000 | 150,000 + | Net credit or debit balance for compliance period | Line 25 | Z | Credits (Debits)| -60,000 | -50,000 + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A-R | Credits | 60,000 | 50,000 + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a | A | Credits | n/a | 60,000 + | Banked credits used to offset outstanding debits - Supplemental Report #1 | Line 26b | Not editable | Credits | n/a | + | Outstanding debit balance | Line 27 | | (Debits) | 0 | 0 + | Part 3 non-compliance penalty payable | Line 28 | | $CAD | | + |-------------------------------------------------------------------------------------|----------|--------------|-----------------|-------------------------------------------|----------------------------------------| + | Banked credits to be returned as a result of supplemental reporting (if applicable) | | R | | | + |-------------------------------------------------------------------------------------|----------|--------------|-----------------|-------------------------------------------|----------------------------------------| + | Report Status (for compliance units conversion) | | | | Submitted (not Accepted) | Accepted + |-------------------------------------------------------------------------------------|----------|--------------|-----------------|-------------------------------------------|----------------------------------------| + | Corresponding Compliance Unit conversion / transaction | | | | | -50,000 + """ + def test_supplemental_report_submission_1_ex2_decreasing_debit_obligation_under_net_debit_position(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 273790701 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(100000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 60000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check + self.assertEqual( + min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 40000) + # exclude the reserved amount as in this case report is not submitted. + self.assertEqual(response.data.get('max_credit_offset_exclude_reserved'), 100000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -60000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 60000.0) + # Create supplemental report #1 + sid1 = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 256678782 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 50000 + payload['summary']['creditsOffsetA'] = 60000 + payload['summary']['creditsOffsetB'] = 0 + self._patch_fs_user_for_compliance_report(payload, sid1) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid1) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid1)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], -50000.0) + self.assertEqual(response.data['summary']['lines']['26'], 50000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 60000.0) + self.assertEqual(response.data['summary']['lines']['27'], 0) + # compliance unit balance check + self.assertEqual( + min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 50000) + + """ + | Scenario 5: Supplemental Report Submission #2 that decreases debit obligation and still in a net debit position overall (Line 25) | + |-----------------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | Units | Example 1 - Initial Submission - Accepted | Example 1 - Supplemental #1 - Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|-------------------------------------------|----------------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | Credits | 100,000 | + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | (Debits) | 170,000 | + | Net credit or debit balance for compliance period | Line 25 | Z | Credits (Debits) | -70,000 | + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A+B+C-R | Credits | 70,000 | + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a | A+B+C | Credits | n/a | + | Banked credits used to offset outstanding debits - Supplemental Report #2 | Line 26b | Not editable | Credits | n/a | + | Outstanding debit balance | Line 27 | | (Debits) | 0 | + | Part 3 non-compliance penalty payable | Line 28 | | $CAD | | + |-------------------------------------------------------------------------------------|----------|--------------|-------------------------------------------|----------------------------------------| + | Banked credits to be returned as a result of supplemental reporting (if applicable) | | R | | | + |-------------------------------------------------------------------------------------|----------|--------------|-------------------------------------------|----------------------------------------| + | Report Status (for compliance units conversion) | | | Accepted | Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|-------------------------------------------|----------------------------------------| + | Corresponding Compliance Unit conversion / transaction | | | | -70,000 | + """ + def test_supplemental_report_submission_2_ex1_decreasing_debit_obligation_under_net_debit_position(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 290902619 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(100000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 70000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 30000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -70000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 70000.0) + # Create supplemental report #1 + sid1 = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 256678782 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 50000 + payload['summary']['creditsOffsetA'] = 70000 + payload['summary']['creditsOffsetB'] = 0 + self._patch_fs_user_for_compliance_report(payload, sid1) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid1) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid1)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], -50000.0) + self.assertEqual(response.data['summary']['lines']['26'], 50000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 70000.0) + self.assertEqual(response.data['summary']['lines']['27'], 0) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 50000) + + """ + | Scenario 5: Supplemental Report Submission #2 that decreases debit obligation and still in a net debit position overall (Line 25) | + |-----------------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | Units | Example 2 - Initial Submission - Accepted | Example 2 - Supplemental #1 - Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|-------------------------------------------|----------------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | 100,000 | 100,000 | + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | 150,000 | 170,000 | + | Net credit or debit balance for compliance period | Line 25 | Z | -50,000 | -70,000 | + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A+B+C-R | 50,000 | 70,000 | + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a | A+B+C | 70,000 | n/a | + | Banked credits used to offset outstanding debits - Supplemental Report #2 | Line 26b | Not editable | 0 | n/a | + | Outstanding debit balance | Line 27 | | 0 | 0 | + | Part 3 non-compliance penalty payable | Line 28 | | | | + |-------------------------------------------------------------------------------------|----------|--------------|-------------------------------------------|----------------------------------------| + | Banked credits to be returned as a result of supplemental reporting (if applicable) | | R | 20,000 | | + |-------------------------------------------------------------------------------------|----------|--------------|-------------------------------------------|----------------------------------------| + | Report Status (for compliance units conversion) | | | Submitted (not Accepted) | Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|-------------------------------------------|----------------------------------------| + | Corresponding Compliance Unit conversion / transaction | | | +20,000 | | + """ + def test_supplemental_report_submission_2_ex2_decreasing_debit_obligation_under_net_debit_position(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 290902619 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(100000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 70000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check + self.assertEqual( + min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 30000) + # exclude the reserved amount as in this case report is not submitted. + self.assertEqual(response.data.get('max_credit_offset_exclude_reserved'), 100000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -70000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 70000.0) + # Create supplemental report #1 + sid1 = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 117933318 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 256678782 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 50000 + payload['summary']['creditsOffsetA'] = 70000 + payload['summary']['creditsOffsetB'] = 0 + self._patch_fs_user_for_compliance_report(payload, sid1) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid1) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid1)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], -50000.0) + self.assertEqual(response.data['summary']['lines']['26'], 50000.0) + self.assertEqual(response.data['summary']['lines']['26A'], 70000.0) + self.assertEqual(response.data['summary']['lines']['27'], 0) + # compliance unit balance check + self.assertEqual( + min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 50000) + + """ + | Scenario 6: Supplemental Report Submission #2 that decreases debit obligation and is now in a net credit position (Line 25) | + |-----------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | | Units | Example 1 - Initial Submission - Accepted | Example 1 - Supplemental #1 - Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|------------------|-------------------------------------------|----------------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | Credits | 80,000 | 105,000 | + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | (Debits) | 100,000 | 100,000 | + | Net credit or debit balance for compliance period | Line 25 | Z | Credits (Debits) | -20,000 | 5,000 | + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A+B+C-R | Credits | 20,000 | 0 | + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a | A+B+C | Credits | n/a | 20,000 | + | Banked credits used to offset outstanding debits - Supplemental Report #2 | Line 26b | Not editable | Credits | n/a | 0 | + | Outstanding debit balance | Line 27 | | (Debits) | | | + | Part 3 non-compliance penalty payable | Line 28 | | $CAD | | | + |-------------------------------------------------------------------------------------|----------|--------------|------------------|-------------------------------------------|----------------------------------------| + | Banked credits to be returned as a result of supplemental reporting (if applicable) | | R | | | 25,000 | + |-------------------------------------------------------------------------------------|----------|--------------|------------------|-------------------------------------------|----------------------------------------| + | Report Status (for compliance units conversion) | | | | Accepted | Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|------------------|-------------------------------------------|----------------------------------------| + | Corresponding Compliance Unit conversion / transaction | | | | -20,000 | +25,000 | + """ + def test_supplemental_report_submission_2_ex1_decreasing_debit_obligation_now_in_net_credit_position(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 94346654 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 171119188 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(100000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 20000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # Successful director acceptance + self._acceptance_from_director(rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 80000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -20000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 20000.0) + # Create supplemental report #1 + sid1 = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 123829984 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 171119188 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 0 + payload['summary']['creditsOffsetA'] = 20000 + payload['summary']['creditsOffsetB'] = 0 + self._patch_fs_user_for_compliance_report(payload, sid1) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid1) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid1)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], 5000.0) + self.assertEqual(response.data['summary']['lines']['26'], 0) + self.assertEqual(response.data['summary']['lines']['26A'], 20000.0) + self.assertEqual(response.data['summary']['lines']['27'], 0) + # compliance unit balance check + self.assertEqual(min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 105000) + + """ + | Scenario 6: Supplemental Report Submission #2 that decreases debit obligation and is now in a net credit position (Line 25) | + |-----------------------------------------------------------------------------------------------------------------------------| + | Part 3 - Low Carbon Fuel Requirement Summary | Line | | Units | Example 2 - Initial Submission - Accepted | Example 2 - Supplemental #1 - Accepted | + |-------------------------------------------------------------------------------------|----------|--------------|------------------|-------------------------------------------|----------------------------------------| + | Total credits from fuel supplied (from Schedule B) | Line 23 | X | Credits | 80,000 | 105,000 + | Total debits from fuel supplied (from Schedule B) | Line 24 | Y | (Debits) | 100,000 | 100,000 + | Net credit or debit balance for compliance period | Line 25 | Z | Credits (Debits) | -20,000 | 5,000 + | Total banked credits used to offset outstanding debits (if applicable) | Line 26 | A+B+C-R | Credits | 20,000 | 0 + | Banked credits used to offset outstanding debits - Previous Reports | Line 26a | A+B+C | Credits | n/a | 20,000 + | Banked credits used to offset outstanding debits - Supplemental Report #2 | Line 26b | Not editable | Credits | n/a | 0 + | Outstanding debit balance | Line 27 | | (Debits) | | + | Part 3 non-compliance penalty payable | Line 28 | | $CAD | | + |-------------------------------------------------------------------------------------|----------|--------------|------------------|-------------------------------------------|----------------------------------------| + | Banked credits to be returned as a result of supplemental reporting (if applicable) | | R | | | 5,000 + |-------------------------------------------------------------------------------------|----------|--------------|------------------|-------------------------------------------|----------------------------------------| + | Report Status (for compliance units conversion) | | | | Submitted (not Accepted) | Accepted + |-------------------------------------------------------------------------------------|----------|--------------|------------------|-------------------------------------------|----------------------------------------| + | Corresponding Compliance Unit conversion / transaction | | | | | +5,000 + """ + def test_supplemental_report_submission_2_ex2_decreasing_debit_obligation_now_in_net_credit_position(self): + rid = self._create_draft_compliance_report() + # patch compliance report info + payload = compliance_unit_initial_payload + payload['status']['fuelSupplierStatus'] = 'Draft' + payload['scheduleB']['records'][0]['quantity'] = 94346654 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 171119188 # debits of fuel supplied (from Schedule B) + self._patch_fs_user_for_compliance_report(payload, rid) + self._add_part3_awards_to_org(100000) + # Submit the compliance report + payload = { + 'status': {'fuelSupplierStatus': 'Submitted'}, + 'summary': { + 'creditsOffset': 20000, + } + } + self._patch_fs_user_for_compliance_report(payload, rid) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=rid)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + # compliance unit balance check + self.assertEqual( + min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 80000) + # exclude the reserved amount as in this case report is not submitted. + self.assertEqual(response.data.get('max_credit_offset_exclude_reserved'), 100000) + self.assertEqual(response.data.get('summary').get('lines').get('25'), -20000.0) + self.assertEqual(response.data.get('summary').get('lines').get('26'), 20000.0) + # Create supplemental report #1 + sid1 = self._create_supplemental_report(rid) + payload = compliance_unit_supplemental_payload + payload['scheduleB']['records'][0]['quantity'] = 123829984 # credits of fuel supplied (from Schedule B) + payload['scheduleB']['records'][1]['quantity'] = 171119188 # debits of fuel supplied (from Schedule B) + payload['summary']['creditsOffset'] = 0 + payload['summary']['creditsOffsetA'] = 20000 + payload['summary']['creditsOffsetB'] = 0 + self._patch_fs_user_for_compliance_report(payload, sid1) + # Successful director acceptance for supplemental report #1 + self._acceptance_from_director(sid1) + # retrieve the compliance report and validate the Summary report fields + response = self.clients['fs_user_1'].get('/api/compliance_reports/{id}'.format(id=sid1)) + self.assertEqual(response.status_code, status.HTTP_200_OK) + self.assertEqual(response.data['summary']['lines']['25'], 5000.0) + self.assertEqual(response.data['summary']['lines']['26'], 0) + self.assertEqual(response.data['summary']['lines']['26A'], 20000) + self.assertEqual(response.data['summary']['lines']['27'], 0) + # compliance unit balance check + self.assertEqual( + min(response.data.get('max_credit_offset'), response.data.get('max_credit_offset_exclude_reserved')), 105000) diff --git a/backend/api/tests/test_credit_trade_admin_adjustment.py b/backend/api/tests/test_credit_trade_admin_adjustment.py new file mode 100644 index 000000000..5998305d6 --- /dev/null +++ b/backend/api/tests/test_credit_trade_admin_adjustment.py @@ -0,0 +1,142 @@ +import datetime +import json + +from rest_framework import status + +from django.db.models import Sum +from api.models.CreditTrade import CreditTrade +from api.tests.base_test_case import BaseTestCase +from api.models.OrganizationBalance import OrganizationBalance + + +class TestAdministrativeAdjustmentOperations(BaseTestCase): + + extra_fixtures = ['test/test_credit_trades.json'] + + def test_administrative_adjustment_positive(self): + """ + Testing positive administrative adjustment + """ + + initial_balance = OrganizationBalance.objects.get( + organization_id=self.users['fs_user_1'].organization.id, + expiration_date=None).validated_credits + + payload = { + 'initiator': self.users['gov_director'].organization.id, + 'respondent': self.users['fs_user_1'].organization.id, + 'numberOfCredits': 10, + 'status': self.statuses['recorded'].id, + 'tradeEffectiveDate': datetime.datetime.today().strftime('%Y-%m-%d'), + 'type': self.credit_trade_types['adminAdjustment'].id + } + response = self.clients['gov_director'].post( + '/api/credit_trades', + content_type='application/json', + data=json.dumps(payload) + ) + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + payload = { + 'initiator': self.users['gov_director'].organization.id, + 'respondent': self.users['fs_user_1'].organization.id, + 'numberOfCredits': 10, + 'status': self.statuses['recorded'].id, + 'tradeEffectiveDate': datetime.datetime.today().strftime('%Y-%m-%d'), + 'type': self.credit_trade_types['adminAdjustment'].id + } + response = self.clients['gov_director'].post( + '/api/credit_trades', + content_type='application/json', + data=json.dumps(payload) + ) + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + + response = self.clients['gov_director'].put('/api/credit_trades/batch_process') + self.assertEqual(response.status_code, status.HTTP_200_OK) + + orgBalances = OrganizationBalance.objects.filter( + organization_id=self.users['fs_user_1'].organization.id, + expiration_date=None) + for org in orgBalances: + print("ORG BALANCE") + print(vars(org)) + + new_balance = OrganizationBalance.objects.get( + organization_id=self.users['fs_user_1'].organization.id, + expiration_date=None).validated_credits + + self.assertEqual(new_balance, initial_balance + 20) + + + def test_administrative_adjustment_negative(self): + """ + Testing negative administrative adjustment + """ + + initial_balance = OrganizationBalance.objects.get( + organization_id=self.users['fs_user_1'].organization.id, + expiration_date=None).validated_credits + print('INITIAL BALANCE') + print(initial_balance) + + payload = { + 'initiator': self.users['gov_director'].organization.id, + 'respondent': self.users['fs_user_1'].organization.id, + 'numberOfCredits': -10, + 'status': self.statuses['recorded'].id, + 'tradeEffectiveDate': datetime.datetime.today().strftime('%Y-%m-%d'), + 'type': self.credit_trade_types['adminAdjustment'].id + } + response = self.clients['gov_director'].post( + '/api/credit_trades', + content_type='application/json', + data=json.dumps(payload) + ) + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + payload = { + 'initiator': self.users['gov_director'].organization.id, + 'respondent': self.users['fs_user_1'].organization.id, + 'numberOfCredits': -10, + 'status': self.statuses['recorded'].id, + 'tradeEffectiveDate': datetime.datetime.today().strftime('%Y-%m-%d'), + 'type': self.credit_trade_types['adminAdjustment'].id + } + response = self.clients['gov_director'].post( + '/api/credit_trades', + content_type='application/json', + data=json.dumps(payload) + ) + self.assertEqual(response.status_code, status.HTTP_201_CREATED) + + response = self.clients['gov_director'].put('/api/credit_trades/batch_process') + self.assertEqual(response.status_code, status.HTTP_200_OK) + + new_balance = OrganizationBalance.objects.get( + organization_id=self.users['fs_user_1'].organization.id, + expiration_date=None).validated_credits + + self.assertEqual(new_balance, initial_balance - 20) + + + def test_administrative_adjustment_insufficient_funds(self): + """ + Testing administrative adjustment with insufficient funds + """ + + # Set a very high negative administrative adjustment value + payload = { + 'initiator': self.users['gov_director'].organization.id, + 'respondent': self.users['fs_user_1'].organization.id, + 'numberOfCredits': -100000000000, + 'status': self.statuses['recorded'].id, + 'tradeEffectiveDate': datetime.datetime.today().strftime('%Y-%m-%d'), + 'type': self.credit_trade_types['adminAdjustment'].id + } + + response = self.clients['gov_director'].post( + '/api/credit_trades', + content_type='application/json', + data=json.dumps(payload) + ) + + self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST) diff --git a/backend/api/tests/test_get_summary.py b/backend/api/tests/test_get_summary.py index f195df338..61b83f0f4 100644 --- a/backend/api/tests/test_get_summary.py +++ b/backend/api/tests/test_get_summary.py @@ -3,6 +3,7 @@ from datetime import datetime from api.serializers.ComplianceReport import ComplianceReportDetailSerializer, CompliancePeriodSerializer from unittest.mock import MagicMock, Mock +from api.models.Organization import Organization class TestComplianceReportDetailSerializer(TestCase): @@ -10,6 +11,8 @@ def setUp(self): self.serializer = ComplianceReportDetailSerializer() self.serializer.compliance_period = CompliancePeriodSerializer() self.serializer.summary = None + self.serializer.supplements = None + self.serializer.organization = Organization.objects.first() self.serializer.schedule_a = MagicMock( net_gasoline_class_transferred=Decimal('10'), diff --git a/backend/api/validators.py b/backend/api/validators.py index 826406c4b..8ece9e093 100644 --- a/backend/api/validators.py +++ b/backend/api/validators.py @@ -3,13 +3,15 @@ from .exceptions import PositiveIntegerException -def CreditTradeNumberOfCreditsValidator(value): +def CreditTradeNumberOfCreditsValidator(value, instance): """ Validates and makes sure that the user doesn't enter 0 or a negative value for the number of - credits + credits, unless the credit_trade_type is 'Administrative Adjustment'. """ - if value <= 0: + if instance.credit_trade_type == 'Administrative Adjustment': + return # Allow any value for Administrative Adjustment + elif value <= 0: raise PositiveIntegerException( "Please enter at least 1 credit." ) diff --git a/backend/api/viewsets/ComplianceReport.py b/backend/api/viewsets/ComplianceReport.py index 8a52314e8..30ebd49e9 100644 --- a/backend/api/viewsets/ComplianceReport.py +++ b/backend/api/viewsets/ComplianceReport.py @@ -1,4 +1,5 @@ import datetime +from decimal import * from django.core.cache import caches, cache from django.db import transaction @@ -23,6 +24,7 @@ ExclusionReportDetailSerializer, ExclusionReportUpdateSerializer, ExclusionReportValidationSerializer from api.services.ComplianceReportService import ComplianceReportService from api.services.ComplianceReportSpreadSheet import ComplianceReportSpreadsheet +from api.services.OrganizationService import OrganizationService from auditable.views import AuditableMixin from api.paginations import BasicPagination from django.db.models import Q, F, Value @@ -114,7 +116,7 @@ def get_queryset(self): qs = qs.annotate(reports_updatedtime=Max('update_timestamp')).order_by('-reports_updatedtime') else: qs = qs.annotate(reports_updatedtime=Max('update_timestamp')).order_by('reports_updatedtime') - + filters = request.data.get('filters') if filters: for filter in filters: @@ -210,12 +212,12 @@ def filter_compliance_status_old(self, qs, value): Q(status__director_status__status='Unreviewed') & Q(status__manager_status__status='Unreviewed') ) - + if 'recommended rejection - analyst'.find(value) != -1 or 'rejection'.find(value) != -1: return qs.filter( Q(status__analyst_status__status='Not Recommended') ) - + if 'recommended acceptance - manager'.find(value) != -1 or 'manager'.find(value) != -1: return qs.filter( Q(status__manager_status__status='Recommended') & @@ -228,13 +230,13 @@ def filter_compliance_status_old(self, qs, value): return qs.filter( Q(status__manager_status__status='Not Recommended') ) - + return qs - + def filter_compliance_status(self, qs, value): query_result = [] for val in value: - if val == 'Accepted' : + if val == 'Accepted' : qs_accepted = qs.filter( Q(status__director_status__status='Accepted') ) @@ -258,7 +260,7 @@ def filter_compliance_status(self, qs, value): query_result.extend(qs_draft) if val == 'For Analyst Review': - qs_analyst = qs.filter( + qs_analyst = qs.filter( Q(status__analyst_status__status='Unreviewed') & Q(status__director_status__status='Unreviewed') & Q(status__fuel_supplier_status__status='Submitted') & @@ -268,12 +270,12 @@ def filter_compliance_status(self, qs, value): if val == 'For Manager Review': qs_manager = qs.filter( - + Q(status__analyst_status__status='Recommended') & Q(status__director_status__status='Unreviewed') & Q(status__manager_status__status='Unreviewed') & Q(status__fuel_supplier_status__status='Submitted') - + ) query_result.extend(qs_manager) qs_man_rej = qs.filter( @@ -283,22 +285,22 @@ def filter_compliance_status(self, qs, value): Q(status__fuel_supplier_status__status='Submitted') ) query_result.extend(qs_man_rej) - + if val == 'For Director Review': qs_director = qs.filter( Q(status__manager_status__status='Recommended') & - Q(status__director_status__status='Unreviewed') + Q(status__director_status__status='Unreviewed') ) query_result.extend(qs_director) qs_dir_rej = qs.filter( Q(status__manager_status__status='Not Recommended') & - Q(status__director_status__status='Unreviewed') + Q(status__director_status__status='Unreviewed') ) query_result.extend(qs_dir_rej) if val == 'awaiting government review': - qs_agr = qs.filter( + qs_agr = qs.filter( Q(status__analyst_status__status='Unreviewed') & Q(status__director_status__status='Unreviewed') & Q(status__fuel_supplier_status__status='Submitted') & @@ -306,9 +308,9 @@ def filter_compliance_status(self, qs, value): ) query_result.extend(qs_agr) - + ids = [i.id for i in query_result] - qs = qs.filter(id__in = ids) + qs = qs.filter(id__in = ids) return qs def filter_supplemental_report_status(self, qs, value): @@ -335,7 +337,7 @@ def filter_supplemental_report_status(self, qs, value): return qs.filter( Q(status__director_status__status='Rejected') ) - + if 'recommended'.find(value) != -1: return qs.filter( (Q(supplements__status__manager_status__status='Recommended') & @@ -386,7 +388,7 @@ def filter_current_status(self, qs, value): def filter_manager_status(self, qs, value): try: - supplemental_reports = ComplianceReport.objects.filter(id__in=value) + supplemental_reports = ComplianceReport.objects.filter(id__in=value) except Exception as e: print(e) return supplemental_reports @@ -399,7 +401,7 @@ def filter_draft(self, latest_supplemental): latest_supplemental = latest_supplemental.filter(~Q(status__fuel_supplier_status__status='Draft')) else: latest_supplemental = latest_supplemental.filter(Q(organization=organization)) - + return latest_supplemental def get_latest_supplemental_reports(self): @@ -546,8 +548,74 @@ def list(self, request, *args, **kwargs): sorted_qs, many=True, context={'request': request}) data = serializer.data cached_page.set(sanitized_cache_key, data, 60 * 15) + return Response(data) + def compliance_to_new_act(self, obj, snapshot): + if int(obj.compliance_period.description) > 2022 and snapshot is not None: + lines = snapshot.get('summary').get('lines') + if lines.get('29A') is None: + previous_transactions = [] + previous_snapshots = [] + current = obj + is_supplemental = False + + if current.supplements: + is_supplemental = True + + available_compliance_unit_balance = OrganizationService.get_max_credit_offset_for_interval( + obj.organization, + obj.update_timestamp + ) + net_compliance_unit_balance = int(lines['25']) + desired_net_credit_balance_change = Decimal(0.0) + if is_supplemental: + while current.supplements is not None: + current = current.supplements + if current.credit_transaction is not None: + previous_transactions.append(current.credit_transaction) + if current.compliance_report_snapshot is not None: + previous_snapshots.append(current.compliance_report_snapshot.snapshot) + + total_previous_reduction = Decimal(0.0) + total_previous_validation = Decimal(0.0) + + for transaction in previous_transactions: + if transaction.type.the_type in ['Credit Validation']: + total_previous_validation += transaction.number_of_credits + if transaction.type.the_type in ['Credit Reduction']: + total_previous_reduction += transaction.number_of_credits + desired_net_credit_balance_change = Decimal(lines['25']) + net_compliance_unit_balance = desired_net_credit_balance_change - \ + (total_previous_validation - total_previous_reduction) + + adjusted_balance = available_compliance_unit_balance + net_compliance_unit_balance + if available_compliance_unit_balance <= 0 and net_compliance_unit_balance < 0: + lines['28'] = int((adjusted_balance * Decimal('-600.00')).max(Decimal(0))) if ( + adjusted_balance < 0) else 0 + lines['29A'] = 0 + total_previous_compliance_units = Decimal(0.0) + for snapshots in previous_snapshots: + if snapshots.get("summary").get("lines") is not None: + total_previous_compliance_units += Decimal(snapshots.get("summary").get("lines").get("25")) + lines['29B'] = Decimal(lines['25']) - total_previous_compliance_units + lines['29C'] = 0 + else: + lines['29A'] = available_compliance_unit_balance + lines['28'] = 0 + if (net_compliance_unit_balance < 0 <= adjusted_balance) or (net_compliance_unit_balance >= 0): + lines['29B'] = net_compliance_unit_balance + elif net_compliance_unit_balance < 0 and adjusted_balance < 0: + lines['29B'] = net_compliance_unit_balance if ( + adjusted_balance > 0) else -available_compliance_unit_balance + lines['28'] = int((adjusted_balance * Decimal('-600.00')).max(Decimal(0))) if ( + adjusted_balance < 0) else 0 + lines['29C'] = lines['29A'] + lines['29B'] + snapshot['summary']['total_payable'] = Decimal(lines['11']) + Decimal(lines['22']) + lines[ + '28'] + snapshot['summary']['lines'] = lines + return snapshot + @action(detail=False, methods=['post']) def paginated(self, request): queryset = self.get_queryset() @@ -565,7 +633,7 @@ def paginated(self, request): serializer = self.get_serializer(queryset, many=True) return Response(serializer.data) - + @action(detail=False, methods=['get'], permission_classes=[AllowAny]) def types(self, request): @@ -600,8 +668,8 @@ def snapshot(self, request, pk=None): # failure to find an object will trigger an exception that is # translated into a 404 snapshot = ComplianceReportSnapshot.objects.get(compliance_report=obj) - - return Response(snapshot.snapshot) + snapshot = self.compliance_to_new_act(obj, snapshot.snapshot) + return Response(snapshot) @action(detail=True, methods=['patch']) def compute_totals(self, request, pk=None): @@ -671,11 +739,14 @@ def xls(self, request, pk=None): if obj.type.the_type == 'Exclusion Report': workbook.add_exclusion_agreement(snapshot['exclusion_agreement']) if obj.type.the_type == 'Compliance Report': + snapshot = self.compliance_to_new_act(obj, snapshot) workbook.add_schedule_a(snapshot['schedule_a']) - workbook.add_schedule_b(snapshot['schedule_b']) + workbook.add_schedule_b(snapshot['schedule_b'], + int(snapshot['compliance_period']['description'])) workbook.add_schedule_c(snapshot['schedule_c']) workbook.add_schedule_d(snapshot['schedule_d']) - workbook.add_schedule_summary(snapshot['summary']) + workbook.add_schedule_summary(snapshot['summary'], + int(snapshot['compliance_period']['description'])) workbook.save(response) @@ -697,7 +768,7 @@ def dashboard(self, request): data = serializer.data cached_page.set(sanitized_cache_key, data, 60 * 15) return Response(data) - + @action(detail=False, methods=['get']) def supplemental(self, request): query_params = request.GET.urlencode() diff --git a/backend/api/viewsets/CreditTrade.py b/backend/api/viewsets/CreditTrade.py index 5742826de..7cc55f8f4 100644 --- a/backend/api/viewsets/CreditTrade.py +++ b/backend/api/viewsets/CreditTrade.py @@ -219,19 +219,23 @@ def batch_process(self, request): """ Call the approve function on multiple Credit Trades """ - status_approved = CreditTradeStatus.objects \ - .get(status="Recorded") + try: + status_approved = CreditTradeStatus.objects \ + .get(status="Recorded") - credit_trades = CreditTrade.objects.filter( - status_id=status_approved.id).order_by('id') + credit_trades = CreditTrade.objects.filter( + status_id=status_approved.id).order_by('id') + + CreditTradeService.validate_credits(credit_trades) - CreditTradeService.validate_credits(credit_trades) + for credit_trade in credit_trades: + credit_trade.update_user_id = request.user.id + CreditTradeService.approve(credit_trade, batch_process=True) + CreditTradeService.dispatch_notifications( + None, credit_trade) - for credit_trade in credit_trades: - credit_trade.update_user_id = request.user.id - CreditTradeService.approve(credit_trade,batch_process=True) - CreditTradeService.dispatch_notifications( - None, credit_trade) + except Exception as e: + return Response(e, status=status.HTTP_400_BAD_REQUEST) return Response({"message": "Approved credit transactions have been processed."}, @@ -248,7 +252,7 @@ def xls(self, request): response['Content-Disposition'] = ( 'attachment; filename="{}.xls"'.format( datetime.datetime.now().strftime( - "BC-LCFS_credit_transactions_%Y-%m-%d") + "BC-LCFS_transactions_%Y-%m-%d") )) credit_trades = self.get_queryset().filter( @@ -275,7 +279,7 @@ def xls(self, request): type="Part3FuelSupplier")) \ .order_by('lower_name') - workbook.add_fuel_suppliers(fuel_suppliers) + workbook.add_fuel_suppliers(fuel_suppliers, include_actions=True) workbook.save(response) diff --git a/backend/api/viewsets/Organization.py b/backend/api/viewsets/Organization.py index 274cdedba..b7bd5819d 100644 --- a/backend/api/viewsets/Organization.py +++ b/backend/api/viewsets/Organization.py @@ -29,6 +29,7 @@ from api.services.CreditTradeService import CreditTradeService from auditable.views import AuditableMixin +from api.services.OrganizationService import OrganizationService cached_page = caches['cached_pages'] @@ -68,7 +69,7 @@ def get_serializer_class(self): @method_decorator(permission_required('VIEW_FUEL_SUPPLIERS')) def list(self, request, *args, **kwargs): """ - Returns a list of Fuel Suppliers + Returns a list of Organizations There are two types of organizations: Government and Fuel Suppliers The function needs to separate the organizations based on type """ @@ -102,19 +103,24 @@ def balance(self, request, pk=None): """ organization = self.get_object() - # Process future effective dates - # This future effective_date feature has been disabled so this - # service method call has been commented out but left here if - # this feature is needed in the future - # CreditTradeService.process_future_effective_dates(organization) + # get the latest balance for the organization balance = OrganizationBalance.objects.get( organization=organization, expiration_date=None) serializer = self.get_serializer(balance) - - return Response(serializer.data) + # access the credit trade data like effective date and compliance period + effective_date_year = datetime.datetime.now().year + max_credit_offset = OrganizationService.get_max_credit_offset(organization, effective_date_year) + max_credit_offset_exclude_reserved = OrganizationService.get_max_credit_offset( + organization, + effective_date_year, + exclude_reserved=True) + data = serializer.data + if data is not None: + data['availableBalance'] = min(max_credit_offset, max_credit_offset_exclude_reserved) + return Response(data) @action(detail=False, methods=['get']) def fuel_suppliers(self, request): @@ -235,7 +241,7 @@ def users(self, request, pk=None): @method_decorator(permission_required('VIEW_FUEL_SUPPLIERS')) def xls(self, request): """ - Exports the Fuel Suppliers as a spreadsheet + Exports the Organizations as a spreadsheet """ response = HttpResponse(content_type='application/ms-excel') response['Content-Disposition'] = ( @@ -251,7 +257,7 @@ def xls(self, request): .order_by('lower_name') workbook = SpreadSheetBuilder() - workbook.add_fuel_suppliers(fuel_suppliers) + workbook.add_fuel_suppliers(fuel_suppliers, include_actions=False) workbook.save(response) return response diff --git a/backend/tfrs/urls.py b/backend/tfrs/urls.py index 8dec8a929..e0b67cadf 100644 --- a/backend/tfrs/urls.py +++ b/backend/tfrs/urls.py @@ -1,7 +1,7 @@ from django.conf.urls import url from django.urls import path, include from django.contrib import admin -import debug_toolbar +# import debug_toolbar from . import views from django.urls import path diff --git a/charts/tfrs-apps/Chart.yaml b/charts/tfrs-apps/Chart.yaml index 9a8fc539e..0b9a41aee 100644 --- a/charts/tfrs-apps/Chart.yaml +++ b/charts/tfrs-apps/Chart.yaml @@ -15,10 +15,10 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 0.1.0 +version: 1.0.0 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to # follow Semantic Versioning. They should reflect the version the application is using. # It is recommended to use it with quotes. -appVersion: "1.16.0" +appVersion: "3.0.0" diff --git a/charts/tfrs-apps/charts/tfrs-backend/Chart.yaml b/charts/tfrs-apps/charts/tfrs-backend/Chart.yaml index 4efbf3ad4..e02f42332 100644 --- a/charts/tfrs-apps/charts/tfrs-backend/Chart.yaml +++ b/charts/tfrs-apps/charts/tfrs-backend/Chart.yaml @@ -15,7 +15,7 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 1.0.0 +version: 0.2.0 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to diff --git a/charts/tfrs-apps/charts/tfrs-backend/templates/_helpers.tpl b/charts/tfrs-apps/charts/tfrs-backend/templates/_helpers.tpl index a64557262..1b1c0be19 100644 --- a/charts/tfrs-apps/charts/tfrs-backend/templates/_helpers.tpl +++ b/charts/tfrs-apps/charts/tfrs-backend/templates/_helpers.tpl @@ -16,10 +16,13 @@ The selector lables: .Release.Name comes from command helm install example: helm install tfrs-backend-dev ... or helm install tfrs-backend-dev-jan ... +.Chart.Name come from the name attribute in Chart.yaml + */}} {{/* -Expand the name of the chart. +Expand the name of the chart. If nameOverride is empty, use .Chart.Name. +Typically no need to assign value to nameOverride, */}} {{- define "tfrs-backend.name" -}} {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} @@ -28,8 +31,7 @@ Expand the name of the chart. {{/* Create a default fully qualified app name. We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). -If release name contains chart name it will be used as a full name. -The .Release.Name is the first parameter of command helm install tfrs-backend +The .Release.Name is the first parameter of command helm install tfrs-backend-dev or tfrs-backend-dev-jan */}} {{- define "tfrs-backend.fullname" -}} {{- .Release.Name }} @@ -60,76 +62,6 @@ Selector labels */}} {{- define "tfrs-backend.selectorLabels" -}} app.kubernetes.io/name: {{ include "tfrs-backend.name" . }} -app.kubernetes.io/instance: {{ .Release.Name }} -{{- end }} - -{{/* -Define the deploymentconfig name -*/}} -{{- define "tfrs-backend.deploymentconfigName" -}} -{{- include "tfrs-backend.fullname" . }} -{{- end }} - -{{/* -Define the deploymentconfig name -*/}} -{{- define "tfrs-backend.imagestreamName" -}} -{{- include "tfrs-backend.fullname" . }} -{{- end }} - -{{/* -Define the service name -*/}} -{{- define "tfrs-backend.serviceName" -}} -{{- include "tfrs-backend.fullname" . }} -{{- end }} - - -{{/* -Define the backend route name -*/}} -{{- define "tfrs-backend.routeName" -}} -{{- include "tfrs-backend.fullname" . }} -{{- end }} - -{{/* -Define the backend admin route name, used by task queue -*/}} -{{- define "tfrs-backend.adminRouteName" -}} -tfrs-backend-admin{{ .Values.suffix }} -{{- end }} - -{{/* -Define the backend static route name, used by task queue -*/}} -{{- define "tfrs-backend.staticRouteName" -}} -tfrs-backend-static{{ .Values.suffix }} -{{- end }} - -{{/* -Define the djangoSecretKey -*/}} -{{- define "tfrs-backend.djangoSecretKey" -}} -{{- randAlphaNum 50 | nospace | b64enc }} +app.kubernetes.io/instance: {{ include "tfrs-backend.fullname" . }} {{- end }} -{{/* -Define the djangoSaltKey -*/}} -{{- define "tfrs-backend.djangoSaltKey" -}} -{{- randAlphaNum 50 | nospace | b64enc }} -{{- end }} - -{{/* -Define the django-secret name -*/}} -{{- define "tfrs-backend.django-secret" -}} -tfrs-django-secret -{{- end }} - -{{/* -Define the django-salt name -*/}} -{{- define "tfrs-backend.django-salt" -}} -tfrs-django-salt -{{- end }} \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-backend/templates/deployment-config.yaml b/charts/tfrs-apps/charts/tfrs-backend/templates/deployment-config.yaml index 2f19fe2e9..94f238e07 100644 --- a/charts/tfrs-apps/charts/tfrs-backend/templates/deployment-config.yaml +++ b/charts/tfrs-apps/charts/tfrs-backend/templates/deployment-config.yaml @@ -3,7 +3,7 @@ apiVersion: apps.openshift.io/v1 metadata: annotations: description: Defines how to deploy the backend application - name: {{ include "tfrs-backend.deploymentconfigName" . }} + name: tfrs-backend{{ .Values.suffix }} labels: {{- include "tfrs-backend.labels" . | nindent 4 }} spec: @@ -30,7 +30,7 @@ spec: from: kind: ImageStreamTag namespace: {{ .Values.namespace }} - name: {{ include "tfrs-backend.name" . }}:{{ .Values.backendImageTagName }} + name: tfrs-backend:{{ .Values.backendImageTagName }} - type: ConfigChange replicas: {{ .Values.replicaCount }} revisionHistoryLimit: 10 @@ -79,7 +79,7 @@ spec: - name: SMTP_SERVER_PORT value: '2500' - name: DATABASE_SERVICE_NAME - value: {{ .Values.env.databaseServiceName }} + value: {{ .Values.databaseServiceHostName }} - name: DATABASE_ENGINE value: postgresql - name: DATABASE_NAME @@ -98,7 +98,7 @@ spec: name: tfrs-patroni-app key: app-db-password - name: POSTGRESQL_SERVICE_HOST - value: {{ .Values.env.postgresqlServiceHost }} + value: {{ .Values.databaseServiceHostName }}.{{ .Values.namespace }}.svc.cluster.local - name: POSTGRESQL_SERVICE_PORT value: '5432' - name: RABBITMQ_USER @@ -107,9 +107,9 @@ spec: name: tfrs-rabbitmq-app key: username - name: RABBITMQ_VHOST - value: tfrs-vhost + value: {{ .Values.rabbitmqVHost }} - name: RABBITMQ_HOST - value: {{ .Values.env.rabbitmqHost }} + value: tfrs-rabbitmq.{{ .Values.namespace }}.svc.cluster.local - name: RABBITMQ_PASSWORD valueFrom: secretKeyRef: @@ -118,7 +118,7 @@ spec: - name: RABBITMQ_PORT value: '5672' - name: MINIO_ENDPOINT - value: {{ .Values.env.minioEndpoint }} + value: tfrs-minio-{{ .Values.envName }}.apps.silver.devops.gov.bc.ca:443 - name: MINIO_USE_SSL value: 'true' - name: DOCUMENTS_API_ENABLED @@ -126,13 +126,13 @@ spec: - name: MINIO_ACCESS_KEY valueFrom: secretKeyRef: - name: {{ .Values.env.minioSecretName }} - key: {{ .Values.env.minioAccessKey}} + name: tfrs-minio-{{ .Values.envName }} + key: MINIO_ACCESS_KEY - name: MINIO_SECRET_KEY valueFrom: secretKeyRef: - name: {{ .Values.env.minioSecretName }} - key: {{ .Values.env.minioSecretKey}} + name: tfrs-minio-{{ .Values.envName }} + key: MINIO_SECRET_KEY - name: FUEL_CODES_API_ENABLED value: '{{ .Values.env.fuelCodesApiEnabled}}' - name: CREDIT_CALCULATION_API_ENABLED @@ -151,8 +151,7 @@ spec: - name: KEYCLOAK_AUDIENCE value: tfrs-on-gold-4308 - name: WELL_KNOWN_ENDPOINT - value: >- - {{ .Values.env.wellKnownEndpoint}} + value: https://{{ .Values.envName }}.loginproxy.gov.bc.ca/auth/realms/standard/.well-known/openid-configuration ports: - containerPort: 8080 protocol: TCP diff --git a/charts/tfrs-apps/charts/tfrs-backend/templates/hpa.yaml b/charts/tfrs-apps/charts/tfrs-backend/templates/hpa.yaml new file mode 100644 index 000000000..abfdbc826 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-backend/templates/hpa.yaml @@ -0,0 +1,32 @@ +{{- if .Values.autoscaling.enabled }} +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: tfrs-backend{{ .Values.suffix }} + labels: + {{- include "tfrs-backend.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: DeploymentConfig + name: tfrs-backend{{ .Values.suffix }} + minReplicas: {{ .Values.autoscaling.minReplicas }} + maxReplicas: {{ .Values.autoscaling.maxReplicas }} + metrics: + {{- if .Values.autoscaling.targetCPUUtilizationPercentage }} + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }} + {{- end }} + {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }} + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }} + {{- end }} +{{- end }} diff --git a/charts/tfrs-apps/charts/tfrs-backend/templates/route.yaml b/charts/tfrs-apps/charts/tfrs-backend/templates/route.yaml new file mode 100644 index 000000000..52105f810 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-backend/templates/route.yaml @@ -0,0 +1,20 @@ +apiVersion: route.openshift.io/v1 +kind: Route +metadata: + name: tfrs-backend{{ .Values.suffix }} + annotations: + haproxy.router.openshift.io/timeout: 1200s + labels: + {{- include "tfrs-backend.labels" . | nindent 4 }} +spec: + host: tfrs-backend{{ .Values.suffix }}.apps.silver.devops.gov.bc.ca + port: + targetPort: web + tls: + insecureEdgeTerminationPolicy: Redirect + termination: edge + to: + kind: Service + name: tfrs-backend{{ .Values.suffix }} + weight: 100 + wildcardPolicy: None diff --git a/charts/tfrs-apps/charts/tfrs-backend/templates/service.yaml b/charts/tfrs-apps/charts/tfrs-backend/templates/service.yaml new file mode 100644 index 000000000..d5c5bc4df --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-backend/templates/service.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: Service +metadata: + name: tfrs-backend{{ .Values.suffix }} + labels: + {{- include "tfrs-backend.labels" . | nindent 4 }} +spec: + type: ClusterIP + ports: + - port: 8080 + targetPort: 8080 + protocol: TCP + name: web + selector: + {{- include "tfrs-backend.selectorLabels" . | nindent 4 }} diff --git a/charts/tfrs-apps/charts/tfrs-backend/values-dev-jan.yaml b/charts/tfrs-apps/charts/tfrs-backend/values-dev-jan.yaml index ab839e9a6..d1508cea6 100644 --- a/charts/tfrs-apps/charts/tfrs-backend/values-dev-jan.yaml +++ b/charts/tfrs-apps/charts/tfrs-backend/values-dev-jan.yaml @@ -2,40 +2,33 @@ # This is a YAML-formatted file. # Declare variables to be passed into your templates. +# backendImageTagName is not in this file, it comes as a argument from the helm command line +# helm template command +# helm template --set backendImageTagName=dev-main-release-jan-2024 -f ./values-dev-jan.yaml tfrs-backend-dev-jan . +# helm -n --set backendImageTagName=dev-main-release-jan-2024 -f ./values-dev-jan.yaml upgrade tfrs-backend-dev-jan . + replicaCount: 1 resources: - # We usually recommend not to specify default resources and to leave this as a conscious - # choice for the user. This also increases chances charts run on environments with little - # resources, such as Minikube. If you do want to specify resources, uncomment the following - # lines, adjust them as necessary, and remove the curly braces after 'resources:'. limits: - cpu: 60m - memory: 60Mi + cpu: 400m + memory: 1200Mi requests: - cpu: 30m - memory: 30Mi + cpu: 200m + memory: 600Mi autoscaling: - enabled: false + enabled: true minReplicas: 1 - maxReplicas: 1 + maxReplicas: 2 targetCPUUtilizationPercentage: 80 # targetMemoryUtilizationPercentage: 90 env: emailSendingEnabled: true djangoDebug: true - databaseServiceName: tfrs-spilo - postgresqlServiceHost: tfrs-spilo.0ab226-dev.svc.cluster.local - rabbitmqHost: tfrs-rabbitmq.0ab226-dev.svc.cluster.local - minioEndpoint: tfrs-minio-test.apps.silver.devops.gov.bc.ca:443 documentsApiEnabled: true - minioSecretName: tfrs-minio-test - minioAccessKey: MINIO_ACCESS_KEY - minioSecretKey: MINIO_SECRET_KEY fuelCodesApiEnabled: true creditCalculationApiEnabled: true complianceReportingApiEnabled: true exclusionReportsApiEnabled: true - wellKnownEndpoint: https://test.loginproxy.gov.bc.ca/auth/realms/standard/.well-known/openid-configuration diff --git a/charts/tfrs-apps/charts/tfrs-backend/values-test-jan.yaml b/charts/tfrs-apps/charts/tfrs-backend/values-test-jan.yaml new file mode 100644 index 000000000..d1508cea6 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-backend/values-test-jan.yaml @@ -0,0 +1,34 @@ +# Default values for itvr-backend. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +# backendImageTagName is not in this file, it comes as a argument from the helm command line +# helm template command +# helm template --set backendImageTagName=dev-main-release-jan-2024 -f ./values-dev-jan.yaml tfrs-backend-dev-jan . +# helm -n --set backendImageTagName=dev-main-release-jan-2024 -f ./values-dev-jan.yaml upgrade tfrs-backend-dev-jan . + +replicaCount: 1 + +resources: + limits: + cpu: 400m + memory: 1200Mi + requests: + cpu: 200m + memory: 600Mi + +autoscaling: + enabled: true + minReplicas: 1 + maxReplicas: 2 + targetCPUUtilizationPercentage: 80 + # targetMemoryUtilizationPercentage: 90 + +env: + emailSendingEnabled: true + djangoDebug: true + documentsApiEnabled: true + fuelCodesApiEnabled: true + creditCalculationApiEnabled: true + complianceReportingApiEnabled: true + exclusionReportsApiEnabled: true diff --git a/charts/tfrs-apps/charts/tfrs-backend/values.yaml b/charts/tfrs-apps/charts/tfrs-backend/values.yaml deleted file mode 100644 index 3826baed2..000000000 --- a/charts/tfrs-apps/charts/tfrs-backend/values.yaml +++ /dev/null @@ -1,82 +0,0 @@ -# Default values for tfrs-backend. -# This is a YAML-formatted file. -# Declare variables to be passed into your templates. - -replicaCount: 1 - -image: - repository: nginx - pullPolicy: IfNotPresent - # Overrides the image tag whose default is the chart appVersion. - tag: "" - -imagePullSecrets: [] -nameOverride: "" -fullnameOverride: "" - -serviceAccount: - # Specifies whether a service account should be created - create: true - # Annotations to add to the service account - annotations: {} - # The name of the service account to use. - # If not set and create is true, a name is generated using the fullname template - name: "" - -podAnnotations: {} - -podSecurityContext: {} - # fsGroup: 2000 - -securityContext: {} - # capabilities: - # drop: - # - ALL - # readOnlyRootFilesystem: true - # runAsNonRoot: true - # runAsUser: 1000 - -service: - type: ClusterIP - port: 80 - -ingress: - enabled: false - className: "" - annotations: {} - # kubernetes.io/ingress.class: nginx - # kubernetes.io/tls-acme: "true" - hosts: - - host: chart-example.local - paths: - - path: / - pathType: ImplementationSpecific - tls: [] - # - secretName: chart-example-tls - # hosts: - # - chart-example.local - -resources: {} - # We usually recommend not to specify default resources and to leave this as a conscious - # choice for the user. This also increases chances charts run on environments with little - # resources, such as Minikube. If you do want to specify resources, uncomment the following - # lines, adjust them as necessary, and remove the curly braces after 'resources:'. - # limits: - # cpu: 100m - # memory: 128Mi - # requests: - # cpu: 100m - # memory: 128Mi - -autoscaling: - enabled: false - minReplicas: 1 - maxReplicas: 100 - targetCPUUtilizationPercentage: 80 - # targetMemoryUtilizationPercentage: 80 - -nodeSelector: {} - -tolerations: [] - -affinity: {} diff --git a/charts/tfrs-apps/charts/tfrs-celery/.helmignore b/charts/tfrs-apps/charts/tfrs-celery/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-celery/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/tfrs-apps/charts/tfrs-celery/Chart.yaml b/charts/tfrs-apps/charts/tfrs-celery/Chart.yaml new file mode 100644 index 000000000..754644731 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-celery/Chart.yaml @@ -0,0 +1,26 @@ +apiVersion: v2 +name: tfrs-celery +description: A Helm chart for Kubernetes + +# A chart can be either an 'application' or a 'library' chart. +# +# Application charts are a collection of templates that can be packaged into versioned archives +# to be deployed. +# +# Library charts provide useful utilities or functions for the chart developer. They're included as +# a dependency of application charts to inject those utilities and functions into the rendering +# pipeline. Library charts do not define any templates and therefore cannot be deployed. +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.1.0 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +# It is recommended to use it with quotes. +appVersion: "3.0.0" + + diff --git a/charts/tfrs-apps/charts/tfrs-celery/templates/_helpers.tpl b/charts/tfrs-apps/charts/tfrs-celery/templates/_helpers.tpl new file mode 100644 index 000000000..95d14c7a9 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-celery/templates/_helpers.tpl @@ -0,0 +1,66 @@ +{{/* + +The labels for all components: + labels: + helm.sh/chart: tfrs-celery-1.0.0 + app.kubernetes.io/name: tfrs-celery + app.kubernetes.io/instance: tfrs-celery-dev or tfrs-celery-dev-jan + app.kubernetes.io/version: "3.0.0" + app.kubernetes.io/managed-by: Helm + +The selector lables: + selector: + app.kubernetes.io/name: tfrs-celery + app.kubernetes.io/instance: tfrs-celery-dev-1977 + +.Release.Name comes from command helm install + example: helm install tfrs-celery-dev ... or helm install tfrs-celery-dev-jan ... + +.Chart.Name come from the name attribute in Chart.yaml + +*/}} + +{{/* +Expand the name of the chart. If nameOverride is empty, use .Chart.Name. +Typically no need to assign value to nameOverride, +*/}} +{{- define "tfrs-celery.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +The .Release.Name is the first parameter of command helm install tfrs-celery-dev or tfrs-celery-dev-jan +*/}} +{{- define "tfrs-celery.fullname" -}} +{{- .Release.Name }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "tfrs-celery.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels: +app.kubernetes.io/managed-by would be Helm +*/}} +{{- define "tfrs-celery.labels" -}} +helm.sh/chart: {{ include "tfrs-celery.chart" . }} +{{ include "tfrs-celery.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "tfrs-celery.selectorLabels" -}} +app.kubernetes.io/name: {{ include "tfrs-celery.name" . }} +app.kubernetes.io/instance: {{ include "tfrs-celery.fullname" . }} +{{- end }} diff --git a/charts/tfrs-apps/charts/tfrs-celery/templates/deployment-config.yaml b/charts/tfrs-apps/charts/tfrs-celery/templates/deployment-config.yaml new file mode 100644 index 000000000..0136244f7 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-celery/templates/deployment-config.yaml @@ -0,0 +1,104 @@ +kind: DeploymentConfig +apiVersion: apps.openshift.io/v1 +metadata: + name: tfrs-celery{{ .Values.suffix }} + labels: + {{- include "tfrs-celery.labels" . | nindent 4 }} +spec: + strategy: + type: Recreate + recreateParams: + timeoutSeconds: 300 + resources: {} + activeDeadlineSeconds: 600 + triggers: + - type: ImageChange + imageChangeParams: + automatic: true + containerNames: + - celery + from: + kind: ImageStreamTag + name: tfrs-celery:{{ .Values.celeryImageTagName }} + - type: ConfigChange + replicas: {{ .Values.replicaCount }} + revisionHistoryLimit: 10 + test: false + selector: + {{- include "tfrs-celery.selectorLabels" . | nindent 4 }} + template: + metadata: + creationTimestamp: null + labels: + {{- include "tfrs-celery.labels" . | nindent 8 }} + spec: + containers: + - name: celery + env: + - name: RABBITMQ_VHOST + value: {{ .Values.rabbitmqVHost }} + - name: RABBITMQ_USER + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: username + - name: RABBITMQ_PASSWORD + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: password + - name: RABBITMQ_HOST + value: tfrs-rabbitmq.{{ .Values.namespace }}.svc.cluster.local + - name: RABBITMQ_PORT + value: '5672' + - name: DATABASE_SERVICE_NAME + value: {{ .Values.databaseServiceHostName }} + - name: DATABASE_ENGINE + value: postgresql + - name: DATABASE_NAME + valueFrom: + secretKeyRef: + name: tfrs-patroni-app + key: app-db-name + - name: DATABASE_USER + valueFrom: + secretKeyRef: + name: tfrs-patroni-app + key: app-db-username + - name: DATABASE_PASSWORD + valueFrom: + secretKeyRef: + name: tfrs-patroni-app + key: app-db-password + - name: MINIO_ENDPOINT + value: tfrs-minio-{{ .Values.envName }}.apps.silver.devops.gov.bc.ca:443 + - name: MINIO_USE_SSL + value: 'true' + - name: MINIO_ACCESS_KEY + valueFrom: + secretKeyRef: + name: tfrs-minio-{{ .Values.envName }} + key: MINIO_ACCESS_KEY + - name: MINIO_SECRET_KEY + valueFrom: + secretKeyRef: + name: tfrs-minio-{{ .Values.envName }} + key: MINIO_SECRET_KEY + - name: MINIO_BUCKET_NAME + value: tfrs + - name: EMAIL_FROM_ADDRESS + value: tfrs@gov.bc.ca + - name: EMAIL_SENDING_ENABLED + value: 'true' + - name: SMTP_SERVER_HOST + value: apps.smtp.gov.bc.ca + resources: +{{ toYaml .Values.resources | indent 12 }} + terminationMessagePath: /dev/termination-log + terminationMessagePolicy: File + imagePullPolicy: Always + restartPolicy: Always + terminationGracePeriodSeconds: 30 + dnsPolicy: ClusterFirst + securityContext: {} + schedulerName: default-scheduler diff --git a/charts/tfrs-apps/charts/tfrs-celery/values-dev-jan.yaml b/charts/tfrs-apps/charts/tfrs-celery/values-dev-jan.yaml new file mode 100644 index 000000000..f69d068ae --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-celery/values-dev-jan.yaml @@ -0,0 +1,19 @@ +# Default values for itvr-backend. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +# backendImageTagName is not in this file, it comes as a argument from the helm command line +# helm template command +# helm template --set backendImageTagName=dev-main-release-jan-2024 -f ./values-dev-jan.yaml tfrs-backend-dev-jan . +# helm -n --set backendImageTagName=dev-main-release-jan-2024 -f ./values-dev-jan.yaml upgrade tfrs-backend-dev-jan . + +replicaCount: 1 + +resources: + limits: + cpu: 200m + memory: 3Gi + requests: + cpu: 100m + memory: 1500Mi + diff --git a/charts/tfrs-apps/charts/tfrs-celery/values-test-jan.yaml b/charts/tfrs-apps/charts/tfrs-celery/values-test-jan.yaml new file mode 100644 index 000000000..f69d068ae --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-celery/values-test-jan.yaml @@ -0,0 +1,19 @@ +# Default values for itvr-backend. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +# backendImageTagName is not in this file, it comes as a argument from the helm command line +# helm template command +# helm template --set backendImageTagName=dev-main-release-jan-2024 -f ./values-dev-jan.yaml tfrs-backend-dev-jan . +# helm -n --set backendImageTagName=dev-main-release-jan-2024 -f ./values-dev-jan.yaml upgrade tfrs-backend-dev-jan . + +replicaCount: 1 + +resources: + limits: + cpu: 200m + memory: 3Gi + requests: + cpu: 100m + memory: 1500Mi + diff --git a/charts/tfrs-apps/charts/tfrs-frontend/.helmignore b/charts/tfrs-apps/charts/tfrs-frontend/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/tfrs-apps/charts/tfrs-frontend/Chart.yaml b/charts/tfrs-apps/charts/tfrs-frontend/Chart.yaml new file mode 100644 index 000000000..2321fa7c9 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/Chart.yaml @@ -0,0 +1,24 @@ +apiVersion: v2 +name: tfrs-frontend +description: A Helm chart for Kubernetes + +# A chart can be either an 'application' or a 'library' chart. +# +# Application charts are a collection of templates that can be packaged into versioned archives +# to be deployed. +# +# Library charts provide useful utilities or functions for the chart developer. They're included as +# a dependency of application charts to inject those utilities and functions into the rendering +# pipeline. Library charts do not define any templates and therefore cannot be deployed. +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.2.0 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +# It is recommended to use it with quotes. +appVersion: "3.0.0" diff --git a/charts/tfrs-apps/charts/tfrs-frontend/templates/_helpers.tpl b/charts/tfrs-apps/charts/tfrs-frontend/templates/_helpers.tpl new file mode 100644 index 000000000..a34119769 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/templates/_helpers.tpl @@ -0,0 +1,66 @@ +{{/* + +The labels for all components: + labels: + helm.sh/chart: tfrs-frontend-1.0.0 + app.kubernetes.io/name: tfrs-frontend + app.kubernetes.io/instance: tfrs-frontend-dev or tfrs-frontend-dev-jan + app.kubernetes.io/version: "3.0.0" + app.kubernetes.io/managed-by: Helm + +The selector lables: + selector: + app.kubernetes.io/name: tfrs-frontend + app.kubernetes.io/instance: tfrs-frontend-dev-1977 + +.Release.Name comes from command helm install + example: helm install tfrs-frontend-dev ... or helm install tfrs-frontend-dev-jan ... + +.Chart.Name come from the name attribute in Chart.yaml + +*/}} + +{{/* +Expand the name of the chart. If nameOverride is empty, use .Chart.Name. +Typically no need to assign value to nameOverride, +*/}} +{{- define "tfrs-frontend.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +The .Release.Name is the first parameter of command helm install tfrs-frontend-dev or tfrs-frontend-dev-jan +*/}} +{{- define "tfrs-frontend.fullname" -}} +{{- .Release.Name }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "tfrs-frontend.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels: +app.kubernetes.io/managed-by would be Helm +*/}} +{{- define "tfrs-frontend.labels" -}} +helm.sh/chart: {{ include "tfrs-frontend.chart" . }} +{{ include "tfrs-frontend.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "tfrs-frontend.selectorLabels" -}} +app.kubernetes.io/name: {{ include "tfrs-frontend.name" . }} +app.kubernetes.io/instance: {{ include "tfrs-frontend.fullname" . }} +{{- end }} diff --git a/charts/tfrs-apps/charts/tfrs-frontend/templates/configmap.yaml b/charts/tfrs-apps/charts/tfrs-frontend/templates/configmap.yaml new file mode 100644 index 000000000..8f3e92ee1 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/templates/configmap.yaml @@ -0,0 +1,28 @@ +kind: ConfigMap +apiVersion: v1 +metadata: + name: tfrs-frontend{{ .Values.suffix }} + creationTimestamp: +data: + features.js: | + window.tfrs_config = { + "keycloak.realm": "standard", + "keycloak.client_id": "{{ .Values.configmap.keycloak.clientId }}", + "keycloak.auth_url": "https://{{ .Values.envName }}.loginproxy.gov.bc.ca/auth", + "keycloak.callback_url": "https://tfrs{{ .Values.suffix }}.apps.silver.devops.gov.bc.ca", + "keycloak.post_logout_url": "https://tfrs{{ .Values.suffix }}.apps.silver.devops.gov.bc.ca", + "keycloak.siteminder_logout_url": "{{ .Values.configmap.keycloak.siteminderLogoutUrl }}", + "debug.enabled": {{ .Values.configmap.debugEnabled }}, + "secure_document_upload.enabled": true, + "secure_document_upload.max_file_size": 50000000, + "fuel_codes.enabled": true, + "keycloak.custom_login": true, + "credit_transfer.enabled": true, + "compliance_reporting.enabled": true, + "compliance_reporting.starting_year": 2017, + "compliance_reporting.create_effective_date": "2019-01-01", + "credit_calculation_api.enabled": true, + "exclusion_reports.enabled": true, + "exclusion_reports.create_effective_date": "2019-01-01", + "api_base": "https://tfrs-backend{{ .Values.suffix }}.apps.silver.devops.gov.bc.ca/api" + }; \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-frontend/templates/deployment-config.yaml b/charts/tfrs-apps/charts/tfrs-frontend/templates/deployment-config.yaml new file mode 100644 index 000000000..d099bf859 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/templates/deployment-config.yaml @@ -0,0 +1,95 @@ + +apiVersion: apps.openshift.io/v1 +kind: DeploymentConfig +metadata: + name: tfrs-frontend{{ .Values.suffix }} + annotations: + description: Defines how to deploy the frontend application + creationTimestamp: null + labels: + {{- include "tfrs-frontend.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.replicaCount }} + revisionHistoryLimit: 10 + selector: + {{- include "tfrs-frontend.selectorLabels" . | nindent 4 }} + strategy: + activeDeadlineSeconds: 600 + recreateParams: + timeoutSeconds: 300 + resources: {} + type: Recreate + template: + metadata: + creationTimestamp: null + labels: + {{- include "tfrs-frontend.labels" . | nindent 8 }} + spec: + volumes: + - name: tfrs-frontend{{ .Values.suffix }} + configMap: + name: tfrs-frontend{{ .Values.suffix }} + containers: + - name: frontend + env: null + image: + imagePullPolicy: IfNotPresent + volumeMounts: + - name: tfrs-frontend{{ .Values.suffix }} + mountPath: /app/static/js/config + ports: + - containerPort: 8080 + protocol: TCP + env: + - name: RABBITMQ_VHOST + value: tfrs{{ .Values.suffix }}-vhost + - name: RABBITMQ_USER + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: username + - name: RABBITMQ_PASSWORD + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: password + - name: RABBITMQ_HOST + value: tfrs-rabbitmq.{{ .Values.namespace }}.svc.cluster.local + - name: RABBITMQ_PORT + value: '5672' + livenessProbe: + failureThreshold: 10 + initialDelaySeconds: 30 + periodSeconds: 10 + successThreshold: 1 + tcpSocket: + port: 8080 + timeoutSeconds: 3 + readinessProbe: + failureThreshold: 10 + initialDelaySeconds: 20 + periodSeconds: 10 + successThreshold: 1 + tcpSocket: + port: 8080 + timeoutSeconds: 3 + resources: +{{ toYaml .Values.resources | indent 12 }} + terminationMessagePolicy: File + dnsPolicy: ClusterFirst + restartPolicy: Always + schedulerName: default-scheduler + securityContext: {} + terminationGracePeriodSeconds: 30 + test: false + triggers: + - imageChangeParams: + automatic: true + containerNames: + - frontend + from: + kind: ImageStreamTag + name: tfrs-frontend:{{ .Values.frontendImageTagName }} + lastTriggeredImage: + type: ImageChange + - type: ConfigChange diff --git a/charts/tfrs-apps/charts/tfrs-frontend/templates/hpa.yaml b/charts/tfrs-apps/charts/tfrs-frontend/templates/hpa.yaml new file mode 100644 index 000000000..85d57587e --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/templates/hpa.yaml @@ -0,0 +1,32 @@ +{{- if .Values.autoscaling.enabled }} +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: tfrs-frontend{{ .Values.suffix }} + labels: + {{- include "tfrs-frontend.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: DeploymentConfig + name: tfrs-frontend{{ .Values.suffix }} + minReplicas: {{ .Values.autoscaling.minReplicas }} + maxReplicas: {{ .Values.autoscaling.maxReplicas }} + metrics: + {{- if .Values.autoscaling.targetCPUUtilizationPercentage }} + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }} + {{- end }} + {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }} + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }} + {{- end }} +{{- end }} \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-frontend/templates/route.yaml b/charts/tfrs-apps/charts/tfrs-frontend/templates/route.yaml new file mode 100644 index 000000000..b2b24b776 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/templates/route.yaml @@ -0,0 +1,22 @@ +{{- if .Values.route.createFrontendRoute }} +apiVersion: route.openshift.io/v1 +kind: Route +metadata: + name: tfrs{{ .Values.suffix }} + annotations: + haproxy.router.openshift.io/timeout: 1200s + labels: + {{- include "tfrs-frontend.labels" . | nindent 4 }} +spec: + host: tfrs{{ .Values.suffix }}.apps.silver.devops.gov.bc.ca + port: + targetPort: web + tls: + insecureEdgeTerminationPolicy: Redirect + termination: edge + to: + kind: Service + name: tfrs-frontend{{ .Values.suffix }} + weight: 100 + wildcardPolicy: None + {{- end }} diff --git a/charts/tfrs-apps/charts/tfrs-frontend/templates/service.yaml b/charts/tfrs-apps/charts/tfrs-frontend/templates/service.yaml new file mode 100644 index 000000000..3cd86a0a9 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/templates/service.yaml @@ -0,0 +1,16 @@ +apiVersion: v1 +kind: Service +metadata: + name: tfrs-frontend{{ .Values.suffix }} + labels: + {{- include "tfrs-frontend.labels" . | nindent 4 }} +spec: + type: ClusterIP + ports: + - port: 8080 + targetPort: 8080 + protocol: TCP + name: web + sessionAffinity: None + selector: + {{- include "tfrs-frontend.selectorLabels" . | nindent 4 }} diff --git a/charts/tfrs-apps/charts/tfrs-frontend/values-dev-jan.yaml b/charts/tfrs-apps/charts/tfrs-frontend/values-dev-jan.yaml new file mode 100644 index 000000000..882ab1112 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/values-dev-jan.yaml @@ -0,0 +1,28 @@ +# Default values for tfrs-frontend. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 + +configmap: + keycloak: + clientId: tfrs-on-gold-4308 + siteminderLogoutUrl: https://logontest7.gov.bc.ca/clp-cgi/logoff.cgi?retnow=1&returl= + debugEnabled: true + +resources: + limits: + cpu: 80m + memory: 120Mi + requests: + cpu: 40m + memory: 60Mi + +route: + createFrontendRoute: true + +autoscaling: + enabled: true + minReplicas: 1 + maxReplicas: 2 + targetCPUUtilizationPercentage: 80 diff --git a/charts/tfrs-apps/charts/tfrs-frontend/values-test-jan.yaml b/charts/tfrs-apps/charts/tfrs-frontend/values-test-jan.yaml new file mode 100644 index 000000000..882ab1112 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-frontend/values-test-jan.yaml @@ -0,0 +1,28 @@ +# Default values for tfrs-frontend. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 + +configmap: + keycloak: + clientId: tfrs-on-gold-4308 + siteminderLogoutUrl: https://logontest7.gov.bc.ca/clp-cgi/logoff.cgi?retnow=1&returl= + debugEnabled: true + +resources: + limits: + cpu: 80m + memory: 120Mi + requests: + cpu: 40m + memory: 60Mi + +route: + createFrontendRoute: true + +autoscaling: + enabled: true + minReplicas: 1 + maxReplicas: 2 + targetCPUUtilizationPercentage: 80 diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/.helmignore b/charts/tfrs-apps/charts/tfrs-notification-server/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/Chart.yaml b/charts/tfrs-apps/charts/tfrs-notification-server/Chart.yaml new file mode 100644 index 000000000..a36160a8b --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/Chart.yaml @@ -0,0 +1,24 @@ +apiVersion: v2 +name: tfrs-notification-server +description: A Helm chart for Kubernetes + +# A chart can be either an 'application' or a 'library' chart. +# +# Application charts are a collection of templates that can be packaged into versioned archives +# to be deployed. +# +# Library charts provide useful utilities or functions for the chart developer. They're included as +# a dependency of application charts to inject those utilities and functions into the rendering +# pipeline. Library charts do not define any templates and therefore cannot be deployed. +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.1.0 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +# It is recommended to use it with quotes. +appVersion: "3.0.0" diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/templates/_helpers.tpl b/charts/tfrs-apps/charts/tfrs-notification-server/templates/_helpers.tpl new file mode 100644 index 000000000..49f5f2f47 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/templates/_helpers.tpl @@ -0,0 +1,67 @@ + +{{/* + +The labels for all components: + labels: + helm.sh/chart: tfrs-backend-1.0.0 + app.kubernetes.io/name: tfrs-backend + app.kubernetes.io/instance: tfrs-backend-dev or tfrs-backend-dev-jan + app.kubernetes.io/version: "3.0.0" + app.kubernetes.io/managed-by: Helm + +The selector lables: + selector: + app.kubernetes.io/name: tfrs-backend + app.kubernetes.io/instance: tfrs-backend-dev-1977 + +.Release.Name comes from command helm install + example: helm install tfrs-backend-dev ... or helm install tfrs-backend-dev-jan ... + +.Chart.Name come from the name attribute in Chart.yaml + +*/}} + +{{/* +Expand the name of the chart. If nameOverride is empty, use .Chart.Name. +Typically no need to assign value to nameOverride, +*/}} +{{- define "tfrs-notification-server.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +The .Release.Name is the first parameter of command helm install tfrs-notification-server-dev or tfrs-notification-server-dev-jan +*/}} +{{- define "tfrs-notification-server.fullname" -}} +{{- .Release.Name }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "tfrs-notification-server.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels: +app.kubernetes.io/managed-by would be Helm +*/}} +{{- define "tfrs-notification-server.labels" -}} +helm.sh/chart: {{ include "tfrs-notification-server.chart" . }} +{{ include "tfrs-notification-server.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "tfrs-notification-server.selectorLabels" -}} +app.kubernetes.io/name: {{ include "tfrs-notification-server.name" . }} +app.kubernetes.io/instance: {{ include "tfrs-notification-server.fullname" . }} +{{- end }} diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/templates/deployment-config.yaml b/charts/tfrs-apps/charts/tfrs-notification-server/templates/deployment-config.yaml new file mode 100644 index 000000000..036bd253f --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/templates/deployment-config.yaml @@ -0,0 +1,93 @@ +kind: DeploymentConfig +apiVersion: apps.openshift.io/v1 +metadata: + name: tfrs-notification-server{{ .Values.suffix }} + creationTimestamp: + labels: + {{- include "tfrs-notification-server.labels" . | nindent 4 }} +spec: + strategy: + type: Recreate + recreateParams: + timeoutSeconds: 600 + resources: {} + activeDeadlineSeconds: 21600 + triggers: + - type: ImageChange + imageChangeParams: + automatic: true + containerNames: + - notification-server + from: + kind: ImageStreamTag + name: tfrs-notification-server:{{ .Values.notificationServerImageTagName }} + lastTriggeredImage: '' + - type: ConfigChange + replicas: 1 + test: false + selector: + {{- include "tfrs-notification-server.selectorLabels" . | nindent 4 }} + template: + metadata: + creationTimestamp: + labels: + {{- include "tfrs-notification-server.labels" . | nindent 8 }} + spec: + containers: + - name: notification-server + image: '' + ports: + - containerPort: 3000 + protocol: TCP + env: + - name: RABBITMQ_HOST + value: tfrs-rabbitmq.{{ .Values.namespace }}.svc.cluster.local + - name: RABBITMQ_VHOST + value: {{ .Values.rabbitmqVHost }} + - name: RABBITMQ_USER + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: username + - name: RABBITMQ_PASSWORD + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: password + - name: NPM_RUN + value: start:notifications + - name: KEYCLOAK_CERTS_URL + value: {{ .Values.keycloak.certsUrl }} + resources: +{{ toYaml .Values.resources | indent 12 }} + livenessProbe: + tcpSocket: + port: 3000 + initialDelaySeconds: 35 + timeoutSeconds: 3 + periodSeconds: 10 + successThreshold: 1 + failureThreshold: 3 + readinessProbe: + tcpSocket: + port: 3000 + initialDelaySeconds: 30 + timeoutSeconds: 3 + periodSeconds: 10 + successThreshold: 1 + failureThreshold: 3 + terminationMessagePath: "/dev/termination-log" + terminationMessagePolicy: File + imagePullPolicy: IfNotPresent + restartPolicy: Always + terminationGracePeriodSeconds: 30 + dnsPolicy: ClusterFirst + securityContext: {} + schedulerName: default-scheduler +status: + latestVersion: 0 + observedGeneration: 0 + replicas: 0 + updatedReplicas: 0 + availableReplicas: 0 + unavailableReplicas: 0 diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/templates/hpa.yaml b/charts/tfrs-apps/charts/tfrs-notification-server/templates/hpa.yaml new file mode 100644 index 000000000..8f6195528 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/templates/hpa.yaml @@ -0,0 +1,32 @@ +{{- if .Values.autoscaling.enabled }} +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: {{ include "tfrs-notification-server.fullname" . }} + labels: + {{- include "tfrs-notification-server.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: {{ include "tfrs-notification-server.fullname" . }} + minReplicas: {{ .Values.autoscaling.minReplicas }} + maxReplicas: {{ .Values.autoscaling.maxReplicas }} + metrics: + {{- if .Values.autoscaling.targetCPUUtilizationPercentage }} + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }} + {{- end }} + {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }} + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }} + {{- end }} +{{- end }} diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/templates/route.yaml b/charts/tfrs-apps/charts/tfrs-notification-server/templates/route.yaml new file mode 100644 index 000000000..2e8d7395c --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/templates/route.yaml @@ -0,0 +1,22 @@ +{{- if .Values.route.createNotificationServerRoute }} +kind: Route +apiVersion: route.openshift.io/v1 +metadata: + name: tfrs-notification-server{{ .Values.suffix }} + creationTimestamp: + labels: + {{- include "tfrs-notification-server.labels" . | nindent 4 }} +spec: + host: tfrs{{ .Values.suffix }}.apps.silver.devops.gov.bc.ca + path: /socket.io + to: + kind: Service + name: tfrs-notification-server${SUFFIX} + weight: 100 + port: + targetPort: notification + tls: + termination: edge + wildcardPolicy: None +status: {} +{{- end }} \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/templates/service.yaml b/charts/tfrs-apps/charts/tfrs-notification-server/templates/service.yaml new file mode 100644 index 000000000..58022384b --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/templates/service.yaml @@ -0,0 +1,17 @@ +kind: Service +apiVersion: v1 +metadata: + name: tfrs-notification-server{{ .Values.suffix }} + creationTimestamp: +spec: + ports: + - name: notification + protocol: TCP + port: 8080 + targetPort: 3000 + selector: + {{- include "tfrs-notification-server.selectorLabels" . | nindent 4 }} + type: ClusterIP + sessionAffinity: None +status: + loadBalancer: {} \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/values-dev-jan.yaml b/charts/tfrs-apps/charts/tfrs-notification-server/values-dev-jan.yaml new file mode 100644 index 000000000..d02f1aa48 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/values-dev-jan.yaml @@ -0,0 +1,25 @@ +# Default values for tfrs-notification-server. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 + +resources: + limits: + cpu: 200m + memory: 240Mi + requests: + cpu: 100m + memory: 120Mi + +route: + createNotificationServerRoute: true + +autoscaling: + enabled: false + minReplicas: 1 + maxReplicas: 2 + targetCPUUtilizationPercentage: 80 + +keycloak: + certsUrl: https://dev.loginproxy.gov.bc.ca/auth/realms/standard/protocol/openid-connect/certs \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-notification-server/values-test-jan.yaml b/charts/tfrs-apps/charts/tfrs-notification-server/values-test-jan.yaml new file mode 100644 index 000000000..8264d87e6 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-notification-server/values-test-jan.yaml @@ -0,0 +1,25 @@ +# Default values for tfrs-notification-server. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 + +resources: + limits: + cpu: 200m + memory: 240Mi + requests: + cpu: 100m + memory: 120Mi + +route: + createNotificationServerRoute: true + +autoscaling: + enabled: false + minReplicas: 1 + maxReplicas: 2 + targetCPUUtilizationPercentage: 80 + +keycloak: + certsUrl: https://test.loginproxy.gov.bc.ca/auth/realms/standard/protocol/openid-connect/certs \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-scan-coordinator/.helmignore b/charts/tfrs-apps/charts/tfrs-scan-coordinator/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-coordinator/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/tfrs-apps/charts/tfrs-scan-coordinator/Chart.yaml b/charts/tfrs-apps/charts/tfrs-scan-coordinator/Chart.yaml new file mode 100644 index 000000000..6d3f252a7 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-coordinator/Chart.yaml @@ -0,0 +1,24 @@ +apiVersion: v2 +name: tfrs-scan-coordinator +description: A Helm chart for Kubernetes + +# A chart can be either an 'application' or a 'library' chart. +# +# Application charts are a collection of templates that can be packaged into versioned archives +# to be deployed. +# +# Library charts provide useful utilities or functions for the chart developer. They're included as +# a dependency of application charts to inject those utilities and functions into the rendering +# pipeline. Library charts do not define any templates and therefore cannot be deployed. +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.1.0 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +# It is recommended to use it with quotes. +appVersion: "1.16.0" diff --git a/charts/tfrs-apps/charts/tfrs-scan-coordinator/templates/_helpers.tpl b/charts/tfrs-apps/charts/tfrs-scan-coordinator/templates/_helpers.tpl new file mode 100644 index 000000000..3a11fdba7 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-coordinator/templates/_helpers.tpl @@ -0,0 +1,66 @@ +{{/* + +The labels for all components: + labels: + helm.sh/chart: tfrs-scan-coordinator-1.0.0 + app.kubernetes.io/name: tfrs-scan-coordinator + app.kubernetes.io/instance: tfrs-scan-coordinator-dev or tfrs-scan-coordinator-dev-jan + app.kubernetes.io/version: "3.0.0" + app.kubernetes.io/managed-by: Helm + +The selector lables: + selector: + app.kubernetes.io/name: tfrs-scan-coordinator + app.kubernetes.io/instance: tfrs-scan-coordinator-dev-1977 + +.Release.Name comes from command helm install + example: helm install tfrs-scan-coordinator-dev ... or helm install tfrs-scan-coordinator-dev-jan ... + +.Chart.Name come from the name attribute in Chart.yaml + +*/}} + +{{/* +Expand the name of the chart. If nameOverride is empty, use .Chart.Name. +Typically no need to assign value to nameOverride, +*/}} +{{- define "tfrs-scan-coordinator.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +The .Release.Name is the first parameter of command helm install tfrs-scan-coordinator-dev or tfrs-scan-coordinator-dev-jan +*/}} +{{- define "tfrs-scan-coordinator.fullname" -}} +{{- .Release.Name }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "tfrs-scan-coordinator.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels: +app.kubernetes.io/managed-by would be Helm +*/}} +{{- define "tfrs-scan-coordinator.labels" -}} +helm.sh/chart: {{ include "tfrs-scan-coordinator.chart" . }} +{{ include "tfrs-scan-coordinator.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "tfrs-scan-coordinator.selectorLabels" -}} +app.kubernetes.io/name: {{ include "tfrs-scan-coordinator.name" . }} +app.kubernetes.io/instance: {{ include "tfrs-scan-coordinator.fullname" . }} +{{- end }} diff --git a/charts/tfrs-apps/charts/tfrs-scan-coordinator/templates/deployment-config.yaml b/charts/tfrs-apps/charts/tfrs-scan-coordinator/templates/deployment-config.yaml new file mode 100644 index 000000000..1b4181c2d --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-coordinator/templates/deployment-config.yaml @@ -0,0 +1,83 @@ +kind: DeploymentConfig +apiVersion: apps.openshift.io/v1 +metadata: + name: tfrs-scan-coordinator{{ .Values.suffix }} + labels: + {{- include "tfrs-scan-coordinator.labels" . | nindent 4 }} +spec: + strategy: + type: Recreate + recreateParams: + timeoutSeconds: 300 + resources: {} + activeDeadlineSeconds: 600 + triggers: + - type: ImageChange + imageChangeParams: + automatic: true + containerNames: + - scan-coordinator + from: + kind: ImageStreamTag + name: tfrs-scan-coordinator:{{ .Values.scanCoordinatorImageTagName }} + - type: ConfigChange + replicas: {{ .Values.replicaCount }} + revisionHistoryLimit: 10 + test: false + selector: + {{- include "tfrs-scan-coordinator.selectorLabels" . | nindent 4 }} + template: + metadata: + creationTimestamp: null + labels: + {{- include "tfrs-scan-coordinator.labels" . | nindent 8 }} + spec: + containers: + - name: scan-coordinator + env: + - name: BYPASS_CLAMAV + value: 'false' + - name: CLAMAV_HOST + value: tfrs-clamav.{{ .Values.namespace }}.svc.cluster.local + - name: CLAMAV_PORT + value: '3310' + - name: AMQP_HOST + value: tfrs-rabbitmq.{{ .Values.namespace }}.svc.cluster.local + - name: AMQP_VHOST + value: {{ .Values.rabbitmqVHost }} + - name: AMQP_PORT + value: '5672' + - name: AMQP_USER + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: username + - name: MINIO_ENDPOINT + value: tfrs-minio-{{ .Values.envName }}.apps.silver.devops.gov.bc.ca + - name: MINIO_USE_SSL + value: 'true' + - name: AMQP_PASSWORD + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: password + - name: MINIO_ACCESS_KEY + valueFrom: + secretKeyRef: + name: tfrs-minio-{{ .Values.envName }} + key: MINIO_ACCESS_KEY + - name: MINIO_SECRET_KEY + valueFrom: + secretKeyRef: + name: tfrs-minio-{{ .Values.envName }} + key: MINIO_SECRET_KEY + resources: +{{ toYaml .Values.resources | indent 12 }} + terminationMessagePath: /dev/termination-log + terminationMessagePolicy: File + imagePullPolicy: Always + restartPolicy: Always + terminationGracePeriodSeconds: 30 + dnsPolicy: ClusterFirst + securityContext: {} + schedulerName: default-scheduler diff --git a/charts/tfrs-apps/charts/tfrs-scan-coordinator/values-dev-jan.yaml b/charts/tfrs-apps/charts/tfrs-scan-coordinator/values-dev-jan.yaml new file mode 100644 index 000000000..df615bbe3 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-coordinator/values-dev-jan.yaml @@ -0,0 +1,12 @@ +# Default values for tfrs-scan-coordinator. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 +resources: + limits: + cpu: 100m + memory: 60Mi + requests: + cpu: 50m + memory: 30Mi \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-scan-coordinator/values-test-jan.yaml b/charts/tfrs-apps/charts/tfrs-scan-coordinator/values-test-jan.yaml new file mode 100644 index 000000000..df615bbe3 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-coordinator/values-test-jan.yaml @@ -0,0 +1,12 @@ +# Default values for tfrs-scan-coordinator. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 +resources: + limits: + cpu: 100m + memory: 60Mi + requests: + cpu: 50m + memory: 30Mi \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-scan-handler/.helmignore b/charts/tfrs-apps/charts/tfrs-scan-handler/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-handler/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/tfrs-apps/charts/tfrs-scan-handler/Chart.yaml b/charts/tfrs-apps/charts/tfrs-scan-handler/Chart.yaml new file mode 100644 index 000000000..a7145fffe --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-handler/Chart.yaml @@ -0,0 +1,24 @@ +apiVersion: v2 +name: tfrs-scan-handler +description: A Helm chart for Kubernetes + +# A chart can be either an 'application' or a 'library' chart. +# +# Application charts are a collection of templates that can be packaged into versioned archives +# to be deployed. +# +# Library charts provide useful utilities or functions for the chart developer. They're included as +# a dependency of application charts to inject those utilities and functions into the rendering +# pipeline. Library charts do not define any templates and therefore cannot be deployed. +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.1.0 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +# It is recommended to use it with quotes. +appVersion: "3.0.0" diff --git a/charts/tfrs-apps/charts/tfrs-scan-handler/templates/_helpers.tpl b/charts/tfrs-apps/charts/tfrs-scan-handler/templates/_helpers.tpl new file mode 100644 index 000000000..3106bcc86 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-handler/templates/_helpers.tpl @@ -0,0 +1,62 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "tfrs-scan-handler.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "tfrs-scan-handler.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "tfrs-scan-handler.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels +*/}} +{{- define "tfrs-scan-handler.labels" -}} +helm.sh/chart: {{ include "tfrs-scan-handler.chart" . }} +{{ include "tfrs-scan-handler.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "tfrs-scan-handler.selectorLabels" -}} +app.kubernetes.io/name: {{ include "tfrs-scan-handler.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Create the name of the service account to use +*/}} +{{- define "tfrs-scan-handler.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "tfrs-scan-handler.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} diff --git a/charts/tfrs-apps/charts/tfrs-scan-handler/templates/deployment-config.yaml b/charts/tfrs-apps/charts/tfrs-scan-handler/templates/deployment-config.yaml new file mode 100644 index 000000000..0d7481446 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-handler/templates/deployment-config.yaml @@ -0,0 +1,82 @@ +kind: DeploymentConfig +apiVersion: apps.openshift.io/v1 +metadata: + name: tfrs-scan-handler{{ .Values.suffix }} + labels: + {{- include "tfrs-scan-handler.labels" . | nindent 4 }} +spec: + strategy: + type: Recreate + recreateParams: + timeoutSeconds: 600 + resources: {} + activeDeadlineSeconds: 21600 + triggers: + - type: ImageChange + imageChangeParams: + automatic: true + containerNames: + - scan-handler + from: + kind: ImageStreamTag + name: tfrs-scan-handler:{{ .Values.scanHandlerImageTagName }} + - type: ConfigChange + replicas: {{ .Values.replicaCount }} + revisionHistoryLimit: 10 + test: false + selector: + {{- include "tfrs-scan-handler.selectorLabels" . | nindent 4 }} + template: + metadata: + creationTimestamp: null + labels: + {{- include "tfrs-scan-handler.selectorLabels" . | nindent 8 }} + spec: + containers: + - name: scan-handler + env: + - name: RABBITMQ_VHOST + value: {{ .Values.rabbitmqVHost }} + - name: RABBITMQ_USER + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: username + - name: RABBITMQ_PASSWORD + valueFrom: + secretKeyRef: + name: tfrs-rabbitmq-app + key: password + - name: RABBITMQ_HOST + value: tfrs-rabbitmq.{{ .Values.namespace }}.svc.cluster.local + - name: RABBITMQ_PORT + value: '5672' + - name: DATABASE_SERVICE_NAME + value: {{ .Values.databaseServiceHostName }} + - name: DATABASE_ENGINE + value: postgresql + - name: DATABASE_NAME + valueFrom: + secretKeyRef: + name: tfrs-patroni-app + key: app-db-name + - name: DATABASE_USER + valueFrom: + secretKeyRef: + name: tfrs-patroni-app + key: app-db-username + - name: DATABASE_PASSWORD + valueFrom: + secretKeyRef: + name: tfrs-patroni-app + key: app-db-password + resources: +{{ toYaml .Values.resources | indent 12 }} + terminationMessagePath: /dev/termination-log + terminationMessagePolicy: File + imagePullPolicy: Always + restartPolicy: Always + terminationGracePeriodSeconds: 30 + dnsPolicy: ClusterFirst + securityContext: {} + schedulerName: default-scheduler diff --git a/charts/tfrs-apps/charts/tfrs-scan-handler/values-dev-jan.yaml b/charts/tfrs-apps/charts/tfrs-scan-handler/values-dev-jan.yaml new file mode 100644 index 000000000..e4228c386 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-handler/values-dev-jan.yaml @@ -0,0 +1,12 @@ +# Default values for tfrs-scan-handler. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 +resources: + limits: + cpu: 50m + memory: 100Mi + requests: + cpu: 25m + memory: 50Mi \ No newline at end of file diff --git a/charts/tfrs-apps/charts/tfrs-scan-handler/values-test-jan.yaml b/charts/tfrs-apps/charts/tfrs-scan-handler/values-test-jan.yaml new file mode 100644 index 000000000..e4228c386 --- /dev/null +++ b/charts/tfrs-apps/charts/tfrs-scan-handler/values-test-jan.yaml @@ -0,0 +1,12 @@ +# Default values for tfrs-scan-handler. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 +resources: + limits: + cpu: 50m + memory: 100Mi + requests: + cpu: 25m + memory: 50Mi \ No newline at end of file diff --git a/charts/tfrs-spilo/values-dev.yaml b/charts/tfrs-spilo/values-dev.yaml index a51647b97..be1c1bfbd 100644 --- a/charts/tfrs-spilo/values-dev.yaml +++ b/charts/tfrs-spilo/values-dev.yaml @@ -1,6 +1,6 @@ spilo: - replicaCount: 2 + replicaCount: 1 credentials: useExistingSecret: true diff --git a/frontend/__tests__/actions/CreditTransfersActions/getCreditTransferType.js b/frontend/__tests__/actions/CreditTransfersActions/getCreditTransferType.js index 2704b217d..ad4f82b9c 100644 --- a/frontend/__tests__/actions/CreditTransfersActions/getCreditTransferType.js +++ b/frontend/__tests__/actions/CreditTransfersActions/getCreditTransferType.js @@ -1,32 +1,38 @@ -import { getCreditTransferType } from '../../../src/actions/creditTransfersActions'; -import { CREDIT_TRANSFER_TYPES } from '../../../src/constants/values'; +import { getCreditTransferType } from '../../../src/actions/creditTransfersActions' +import { CREDIT_TRANSFER_TYPES } from '../../../src/constants/values' test('getCreditTransferType should return a display value for Validation', () => { - const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.validation.id); + const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.validation.id) - expect('Validation').toEqual(data); -}); + expect('Assessment').toEqual(data) +}) test('getCreditTransferType should return a display value for Reduction', () => { - const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.retirement.id); + const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.retirement.id) - expect('Reduction').toEqual(data); -}); + expect('Assessment').toEqual(data) +}) test('getCreditTransferType should return a display value for Part 3 Award', () => { - const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.part3Award.id); + const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.part3Award.id) - expect('Part 3 Award').toEqual(data); -}); + expect('Part 3 Award').toEqual(data) +}) + +test('getCreditTransferType should return a display value for Administrative Adjustment', () => { + const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.adminAdjustment.id) + + expect('Administrative Adjustment').toEqual(data) +}) test('getCreditTransferType should return a display value for Sell', () => { - const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.sell.id); + const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.sell.id) - expect('Credit Transfer').toEqual(data); -}); + expect('Transfer').toEqual(data) +}) test('getCreditTransferType should return a display value for Buy', () => { - const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.buy.id); + const data = getCreditTransferType(CREDIT_TRANSFER_TYPES.buy.id) - expect('Credit Transfer').toEqual(data); -}); + expect('Transfer').toEqual(data) +}) diff --git a/frontend/__tests__/actions/CreditTransfersActions/prepareCreditTransfer.js b/frontend/__tests__/actions/CreditTransfersActions/prepareCreditTransfer.js index bc9772a08..a69107dc4 100644 --- a/frontend/__tests__/actions/CreditTransfersActions/prepareCreditTransfer.js +++ b/frontend/__tests__/actions/CreditTransfersActions/prepareCreditTransfer.js @@ -1,5 +1,5 @@ -import { prepareCreditTransfer } from '../../../src/actions/creditTransfersActions'; -import { CREDIT_TRANSFER_STATUS, CREDIT_TRANSFER_TYPES, DEFAULT_ORGANIZATION } from '../../../src/constants/values'; +import { prepareCreditTransfer } from '../../../src/actions/creditTransfersActions' +import { CREDIT_TRANSFER_STATUS, CREDIT_TRANSFER_TYPES, DEFAULT_ORGANIZATION } from '../../../src/constants/values' test('prepareCreditTransfer should return the right data for Credit Transfers (Sell)', () => { const data = prepareCreditTransfer({ @@ -13,7 +13,7 @@ test('prepareCreditTransfer should return the right data for Credit Transfers (S tradeEffectiveDate: '2018-01-01', transferType: CREDIT_TRANSFER_TYPES.sell.id, zeroDollarReason: '' - }); + }) expect({ compliancePeriod: null, @@ -24,8 +24,8 @@ test('prepareCreditTransfer should return the right data for Credit Transfers (S tradeEffectiveDate: '2018-01-01', type: CREDIT_TRANSFER_TYPES.sell.id, zeroReason: '' - }).toEqual(data); -}); + }).toEqual(data) +}) test('prepareCreditTransfer should return the right data for Part 3 Award', () => { const data = prepareCreditTransfer({ @@ -39,7 +39,7 @@ test('prepareCreditTransfer should return the right data for Part 3 Award', () = tradeEffectiveDate: '2018-01-01', transferType: CREDIT_TRANSFER_TYPES.part3Award.id, zeroDollarReason: '' - }); + }) expect({ compliancePeriod: null, @@ -50,8 +50,60 @@ test('prepareCreditTransfer should return the right data for Part 3 Award', () = tradeEffectiveDate: '2018-01-01', type: CREDIT_TRANSFER_TYPES.part3Award.id, zeroReason: '' - }).toEqual(data); -}); + }).toEqual(data) +}) + +test('prepareCreditTransfer should return the right data for a positive Administrative Adjustment', () => { + const data = prepareCreditTransfer({ + creditsFrom: { + id: 0 + }, + creditsTo: { + id: 5 + }, + numberOfCredits: 100, + tradeEffectiveDate: '2018-01-01', + transferType: CREDIT_TRANSFER_TYPES.adminAdjustment.id, + zeroDollarReason: '' + }) + + expect({ + compliancePeriod: null, + initiator: DEFAULT_ORGANIZATION.id, + numberOfCredits: 100, + respondent: 5, + status: CREDIT_TRANSFER_STATUS.recorded.id, + tradeEffectiveDate: '2018-01-01', + type: CREDIT_TRANSFER_TYPES.adminAdjustment.id, + zeroReason: '' + }).toEqual(data) +}) + +test('prepareCreditTransfer should return the right data for a negative Administrative Adjustment', () => { + const data = prepareCreditTransfer({ + creditsFrom: { + id: 0 + }, + creditsTo: { + id: 5 + }, + numberOfCredits: -100, + tradeEffectiveDate: '2018-01-01', + transferType: CREDIT_TRANSFER_TYPES.adminAdjustment.id, + zeroDollarReason: '' + }) + + expect({ + compliancePeriod: null, + initiator: DEFAULT_ORGANIZATION.id, + numberOfCredits: -100, + respondent: 5, + status: CREDIT_TRANSFER_STATUS.recorded.id, + tradeEffectiveDate: '2018-01-01', + type: CREDIT_TRANSFER_TYPES.adminAdjustment.id, + zeroReason: '' + }).toEqual(data) +}) test('prepareCreditTransfer should return the right data for Validation', () => { const data = prepareCreditTransfer({ @@ -65,7 +117,7 @@ test('prepareCreditTransfer should return the right data for Validation', () => tradeEffectiveDate: '2018-01-01', transferType: CREDIT_TRANSFER_TYPES.validation.id, zeroDollarReason: '' - }); + }) expect({ compliancePeriod: null, @@ -76,8 +128,8 @@ test('prepareCreditTransfer should return the right data for Validation', () => tradeEffectiveDate: '2018-01-01', type: CREDIT_TRANSFER_TYPES.validation.id, zeroReason: '' - }).toEqual(data); -}); + }).toEqual(data) +}) test('prepareCreditTransfer should return the right data for Reduction', () => { const data = prepareCreditTransfer({ @@ -91,7 +143,7 @@ test('prepareCreditTransfer should return the right data for Reduction', () => { tradeEffectiveDate: '2018-01-01', transferType: CREDIT_TRANSFER_TYPES.retirement.id, zeroDollarReason: '' - }); + }) expect({ compliancePeriod: null, @@ -102,5 +154,5 @@ test('prepareCreditTransfer should return the right data for Reduction', () => { tradeEffectiveDate: '2018-01-01', type: CREDIT_TRANSFER_TYPES.retirement.id, zeroReason: '' - }).toEqual(data); -}); + }).toEqual(data) +}) diff --git a/frontend/package.json b/frontend/package.json index f0c4b0fe2..767ff001e 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,6 +1,6 @@ { "name": "tfrs", - "version": "2.13.0", + "version": "3.0.0", "dependencies": { "@babel/eslint-parser": "^7.19.1", "@babel/plugin-proposal-object-rest-spread": "^7.20.7", diff --git a/frontend/src/actions/creditTransfersActions.js b/frontend/src/actions/creditTransfersActions.js index f6d4d4266..c218aeda1 100644 --- a/frontend/src/actions/creditTransfersActions.js +++ b/frontend/src/actions/creditTransfersActions.js @@ -4,7 +4,7 @@ import ActionTypes from '../constants/actionTypes/CreditTransfers' import ReducerTypes from '../constants/reducerTypes/CreditTransfers' import * as Routes from '../constants/routes' import { CREDIT_TRANSFER_STATUS, CREDIT_TRANSFER_TYPES, DEFAULT_ORGANIZATION } from '../constants/values' - +import moment from 'moment-timezone' /* * Credit Transfers */ @@ -18,16 +18,19 @@ export const getCreditTransfers = () => (dispatch) => { }) } -export const getCreditTransferType = (typeId) => { +export const getCreditTransferType = (typeId, updateTimestamp=null) => { + const jan2024Timestamp = moment('2024-01-01'); switch (typeId) { case CREDIT_TRANSFER_TYPES.validation.id: - return 'Validation' + return updateTimestamp && moment(updateTimestamp).isAfter(jan2024Timestamp) ? 'Assessment' : 'Validation'; case CREDIT_TRANSFER_TYPES.retirement.id: - return 'Reduction' + return updateTimestamp && moment(updateTimestamp).isAfter(jan2024Timestamp) ? 'Assessment' : 'Reduction'; case CREDIT_TRANSFER_TYPES.part3Award.id: - return 'Part 3 Award' + return 'Initiative Agreement' + case CREDIT_TRANSFER_TYPES.adminAdjustment.id: + return 'Administrative Adjustment' default: - return 'Credit Transfer' + return 'Transfer' } } @@ -61,6 +64,11 @@ export const prepareCreditTransfer = (fields) => { data.initiator = DEFAULT_ORGANIZATION.id data.respondent = fields.creditsFrom.id + break + case CREDIT_TRANSFER_TYPES.adminAdjustment.id.toString(): + data.initiator = DEFAULT_ORGANIZATION.id + data.respondent = fields.creditsTo.id + break default: data.initiator = (fields.creditsFrom.id > 0) diff --git a/frontend/src/actions/organizationActions.js b/frontend/src/actions/organizationActions.js index 77c774c3a..c6535172c 100644 --- a/frontend/src/actions/organizationActions.js +++ b/frontend/src/actions/organizationActions.js @@ -54,6 +54,35 @@ const getMyOrganizationMembers = () => (dispatch) => { }) } +const getOrganizationBalance = id => (dispatch) => { + dispatch(getOrganizationBalanceRequest()) + + axios.get(`${Routes.BASE_URL}${Routes.ORGANIZATIONS_API}/${id}/balance`) + .then((response) => { + dispatch(getOrganizationBalanceSuccess(response.data)) + }).catch((error) => { + dispatch(getOrganizationBalanceError(error.response)) + }) +} + +const getOrganizationBalanceError = error => ({ + errorMessage: error, + name: ReducerTypes.ERROR_ORGANIZATION_BALANCE_REQUEST, + type: ActionTypes.ERROR +}) + +const getOrganizationBalanceSuccess = balance => ({ + details: balance, + name: ReducerTypes.RECEIVE_ORGANIZATION_BALANCE_REQUEST, + receivedAt: Date.now(), + type: ActionTypes.RECEIVE_ORGANIZATION_BALANCE +}) + +const getOrganizationBalanceRequest = () => ({ + name: ReducerTypes.GET_ORGANIZATION_BALANCE_REQUEST, + type: ActionTypes.GET_ORGANIZATION_BALANCE +}) + const getOrganization = id => (dispatch) => { dispatch(getOrganizationRequest()) @@ -200,5 +229,5 @@ const updateOrganizationSuccess = response => ({ export { getFuelSuppliers, getMyOrganization, getMyOrganizationMembers, getOrganization, getOrganizationMembers, getOrganizations, - addOrganization, updateOrganization + addOrganization, updateOrganization, getOrganizationBalance } diff --git a/frontend/src/admin/historical_data_entry/components/HistoricalConfirmationTable.js b/frontend/src/admin/historical_data_entry/components/HistoricalConfirmationTable.js new file mode 100644 index 000000000..f9574017c --- /dev/null +++ b/frontend/src/admin/historical_data_entry/components/HistoricalConfirmationTable.js @@ -0,0 +1,131 @@ +import React, { useEffect, useState } from 'react' +import { connect } from 'react-redux' +import PropTypes from 'prop-types' +import { bindActionCreators } from 'redux' +import ReactDataSheet from 'react-datasheet' +import Loading from '../../../app/components/Loading' + +import { getOrganizationBalance } from '../../../actions/organizationActions' + +const HistoricalConfirmationTable = props => { + const { item, organizationBalance, getOrganizationBalance } = props + const [availableBalance, setAvailableBalance] = useState(0) + + useEffect(() => { + // Check if organization balance is fetching + if (!organizationBalance.isFetching) { + getOrganizationBalance(item.creditsTo.id) + } + }, [item.creditsTo.id]) + + // Check if organization balance is still fetching, if so, display a loading indicator + if (organizationBalance.isFetching) { + return + } + if (organizationBalance.details && + organizationBalance.details.availableBalance !== availableBalance && + item.creditsTo.id === organizationBalance.details.organization) { + setAvailableBalance(organizationBalance.details.availableBalance) + } + + function decimalViewer (digits = 2) { + return cell => Number(cell.value).toFixed(digits) + .toString().replace(/(\d)(?=(\d{3})+(?!\d))/g, '$1,') + } + + function buildGrid (item) { + const credits = item.numberOfCredits + const offset = availableBalance + credits + + const nonCompliancePenalty = (credits < 0 && offset < 0) ? offset * -600 : 0 + const balanceChange = (offset < 0) ? (availableBalance * -1) : credits + const balanceAfterTransaction = availableBalance + balanceChange + + const grid = [ + [{ + className: 'text', + readOnly: true, + value: 'Transaction' + }, { + className: 'number', + readOnly: true, + valueViewer: decimalViewer(0), + value: credits // credits + }], [{ + className: 'text', + readOnly: true, + value: 'Current compliance unit balance' + }, { + className: 'number', + readOnly: true, + valueViewer: decimalViewer(0), + value: availableBalance // balance + }], [{ + className: 'text', + readOnly: true, + value: 'Compliance unit balance change from this transaction' + }, { + className: 'number', + readOnly: true, + valueViewer: decimalViewer(0), + value: balanceChange // balance change from this transaction + }], [{ + className: 'text', + readOnly: true, + value: 'Compliance unit balance after this transaction is committed' + }, { + className: 'number', + readOnly: true, + valueViewer: decimalViewer(0), + value: balanceAfterTransaction // balance after transaction + }], [{ + className: 'text', + readOnly: true, + value: `Non-compliance penalty payable, if applicable (${offset * -1} * $600 CAD per unit)` + }, { + className: 'number', + readOnly: true, + valueViewer: decimalViewer(0), + value: nonCompliancePenalty // balance after transaction + }] + ] + if (nonCompliancePenalty <= 0) { + grid[4][0].className = 'hidden' + grid[4][1].className = 'hidden' + } + return grid + } + + return ( + <> +

{item.creditsTo.name} compliance unit balance will change as follows:

+ cell.value} + /> + + ) +} + +HistoricalConfirmationTable.propTypes = { + item: PropTypes.object.isRequired, + getOrganizationBalance: PropTypes.func.isRequired, + organizationBalance: PropTypes.shape({ + details: PropTypes.object, + isFetching: PropTypes.bool + }).isRequired +} + +const mapStateToProps = state => ({ + organizationBalance: { + details: state.rootReducer.organizationBalanceRequest.details, + isFetching: state.rootReducer.organizationBalanceRequest.isFetching + } +}) + +const mapDispatchToProps = dispatch => ({ + getOrganizationBalance: bindActionCreators(getOrganizationBalance, dispatch) +}) + +export default connect(mapStateToProps, mapDispatchToProps)(HistoricalConfirmationTable) diff --git a/frontend/src/admin/historical_data_entry/components/HistoricalDataEntryFormDetails.js b/frontend/src/admin/historical_data_entry/components/HistoricalDataEntryFormDetails.js index 1394ac778..657096cd8 100644 --- a/frontend/src/admin/historical_data_entry/components/HistoricalDataEntryFormDetails.js +++ b/frontend/src/admin/historical_data_entry/components/HistoricalDataEntryFormDetails.js @@ -39,17 +39,19 @@ const HistoricalDataEntryFormDetails = props => (
-
@@ -144,6 +147,7 @@ const HistoricalDataEntryFormDetails = props => (