Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: make terragrunt-fetch-dependency-output-from-state to work with not applied dependencies #3350

Merged

Conversation

rodrigorfk
Copy link
Contributor

@rodrigorfk rodrigorfk commented Aug 19, 2024

Description

Fixes #3349 and #2261.

If you enable the --terragrunt-fetch-dependency-output-from-state feature flag and runs a terragrunt init on a module that depends on a dependency that was not yet applied, terragrunt will fail, even if you are using mock_outputs. This fix is treating the returned error and checking if a NoSuchKey error was produced, allowing terragrunt to proceed if that was the case.

TODOs

Read the Gruntwork contribution guidelines.

  • Update the docs.
  • Run the relevant tests successfully, including pre-commit checks.
  • Ensure any 3rd party code adheres with our license policy or delete this line if its not applicable.
  • Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Fix --terragrunt-fetch-dependency-output-from-state feature flag and allows it to work with not applied dependencies

Migration Guide

…h not applied dependencies

Signed-off-by: Rodrigo Fior Kuntzer <[email protected]>
@denis256
Copy link
Member

Looks like TestTerragruntMockOutputsFromRemoteState is failing

=== RUN   TestTerragruntMockOutputsFromRemoteState
    integration_test.go:4513: Copying fixture-output-from-remote-state to /tmp/terragrunt-test4260963045
    integration_test.go:4467: [terragrunt apply --terragrunt-fetch-dependency-output-from-state --auto-approve --terragrunt-non-interactive --terragrunt-working-dir /tmp/terragrunt-test4260963045/fixture-output-from-remote-state/env1/app1]

�[0m�[1mInitializing the backend...�[0m
�[0m�[32m
Successfully configured the backend "s3"! OpenTofu will automatically
use this backend unless the backend configuration changes.�[0m

�[0m�[1mInitializing provider plugins...�[0m

�[0m�[1m�[32mOpenTofu has been successfully initialized!�[0m�[32m�[0m
�[0m�[32m
You may now begin working with OpenTofu. Try running "tofu plan" to see
any changes that are required for your infrastructure. All OpenTofu commands
should now work.

If you ever set or change modules or backend configuration for OpenTofu,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.�[0m

Changes to Outputs:
  �[32m+�[0m�[0m app1_text = "app1 output"

You can apply this plan to save these new output values to the OpenTofu
state, without changing any real infrastructure.
�[0m�[1m�[32m
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
�[0m�[0m�[1m�[32m
Outputs:

�[0mapp1_text = "app1 output"
    integration_test.go:4467: [terragrunt init --terragrunt-fetch-dependency-output-from-state --terragrunt-non-interactive --terragrunt-working-dir /tmp/terragrunt-test4260963045/fixture-output-from-remote-state/env1/app2]
    integration_test.go:5371: 
        	Error Trace:	/home/circleci/project/test/integration_test.go:5371
        	Error:      	Should be true
        	Test:       	TestTerragruntMockOutputsFromRemoteState
    integration_test.go:4824: Deleting test s3 bucket terragrunt-test-bucket-eseh5d
--- FAIL: TestTerragruntMockOutputsFromRemoteState (5.92s)

@rodrigorfk
Copy link
Contributor Author

Thanks for the update @denis256 , I added a check that is terraform only and therefore it fails with tofu, I've amended the PR and fixed it, could you please run the tests once again? thanks in advance.

rootTerragruntConfigPath := util.JoinPath(tmpEnvPath, TEST_FIXTURE_OUTPUT_FROM_REMOTE_STATE, config.DefaultTerragruntConfigPath)
copyTerragruntConfigAndFillPlaceholders(t, rootTerragruntConfigPath, rootTerragruntConfigPath, s3BucketName, "not-used", "not-used")

environmentPath := fmt.Sprintf("%s/%s/env1", tmpEnvPath, TEST_FIXTURE_OUTPUT_FROM_REMOTE_STATE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think filepath.Join() will be more cross-platform

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually copied that code from the TestTerragruntOutputFromRemoteState test above, but sure, make sense and thanks for the suggestion, I've fixed it.

stdout bytes.Buffer
stderr bytes.Buffer
)
runTerragruntRedirectOutput(t, fmt.Sprintf("terragrunt init --terragrunt-fetch-dependency-output-from-state --terragrunt-non-interactive --terragrunt-working-dir %s/app2", environmentPath), &stdout, &stderr)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runTerragruntCommandWithOutput() - will also print output to terminal, will be helpful to debug why it is crashing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the suggestion, I've updated it.

Signed-off-by: Rodrigo Fior Kuntzer <[email protected]>
Copy link

sonarcloud bot commented Aug 19, 2024

@denis256 denis256 merged commit 5867027 into gruntwork-io:master Aug 20, 2024
4 checks passed
akrantz01 pushed a commit to akrantz01/homelab that referenced this pull request Aug 20, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
|
[gruntwork-io/terragrunt](https://github.com/gruntwork-io/terragrunt)
| patch | `v0.66.8` -> `v0.66.9` |
|
[gruntwork-io/terragrunt](https://github.com/gruntwork-io/terragrunt)
| patch | `0.66.8` -> `0.66.9` |

---

### Release Notes

<details>
<summary>gruntwork-io/terragrunt (gruntwork-io/terragrunt)</summary>

###
[`v0.66.9`](https://github.com/gruntwork-io/terragrunt/releases/tag/v0.66.9)

[Compare
Source](https://github.com/gruntwork-io/terragrunt/compare/v0.66.8...v0.66.9)

#### Updated CLI args, config attributes and blocks

-   `--terragrunt-fetch-dependency-output-from-state`

#### Description

- Fixed the `--terragrunt-fetch-dependency-output-from-state` feature
flag to work correctly with non-applied dependencies.

#### Special thanks

Special thanks to [@&#8203;rodrigorfk](https://github.com/rodrigorfk)
for their contribution!

#### Related links

-
[gruntwork-io/terragrunt#3350

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View the
[repository job
log](https://developer.mend.io/github/akrantz01/homelab).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC4yNi4xIiwidXBkYXRlZEluVmVyIjoiMzguMjYuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants