Conditional operator doesn't short-circuit evaluation based on result

[apparentlymart]

To avoid extensive refactoring of the evaluator while implementing the conditional syntax, we accepted the limitation that it would evaluate both "sides" and then discard the inappropriate one when given syntax like this:

foo ? bar : baz

This is in most cases just a small waste of compute to evaluate a result that will never be used, but it's troublesome in cases where one of the expressions can be evaluated without error only if its associated condition state is true. For example:

length(foo) > 0 ? foo[0] : "default"

This doesn't work as intended because we fail on trying to index into the empty list foo in spite of the conditional.

The correct behavior is for the evaluator to evaluate the boolean expression first and then visit only one of the given expressions.

[tomdavidson]

Not complaining, just noting that it is more than cpu cycles. For example, my module was intended to have var.accounts optional.

  group = "${lookup(var.accounts, "unset", "") != ""
    ? element(aws_iam_group.default.*.id, count.index)
    : aws_iam_group.default.id}"

aws_iam_group.default does not exist when var.accounts is set.

[Quentin-M]

Completely breaks patterns such as:

ca_cert = "${length(var.tectonic_ca_cert_path) > 0 ? file(var.tectonic_ca_cert_path) : ""}"

or

ca_cert = "${var.ca_cert == "" ? tls_self_signed_cert.ca.cert_pem : var.ca_cert}"

Which is not so great because it means there's literally no way as far as I know to support generating OR importing a certificate. Except if you always create the CA resource, use heredocs to specify the existing PEM-Encoded CA cert in tfvars then use either. Hacks all over the place.

Is there a timeline for that one? It's quite incapacitating.

[Quentin-M]

Actually, here is a wacky workaround for the second use-case:

ca_cert = "${var.ca_cert == "" ? join(" ", tls_self_signed_cert.ca.*.cert_pem) : var.ca_cert}"

[fatmcgav]

Just to add a scenario that I've just attempted that's hit this issue:

# CloudInit config
data "template_cloudinit_config" "instance_userdata" {
  count = "${length(var.userdata_template) > 0 ? var.count * length(split(",", lookup(var.common, "azs"))): 0}"

  /* count = "${var.count * length(split(",", lookup(var.common, "azs")))}" */

  part {
    content_type = "text/cloud-config"
    content      = "${element(data.template_file.instance_userdata.*.rendered, count.index)}"
  }
}

# Create required instance(s)
resource "openstack_compute_instance_v2" "instance" {
  count               = "${var.count * length(split(",", lookup(var.common, "azs")))}"
  ... snipped for brevity...

  user_data = "${length(var.userdata_template) > 0 ? element(data.template_cloudinit_config.instance_userdata.*.rendered, count.index) : ""}"
  ... snip ...
}

You can see my commented count in the template_cloudinit_config resource, which worked fine when if I'm always providing a userdata_template, however I want it to be optional, hence the user of length.

However the above code fails with:

 Error running plan: 1 error(s) occurred:

* module.instance_test.openstack_compute_instance_v2.instance: 2 error(s) occurred:

* module.instance_test.openstack_compute_instance_v2.instance[0]: element: element() may not be used with an empty list in:

${length(var.userdata_template) > 0 ? element(data.template_cloudinit_config.instance_userdata.*.rendered, count.index) : ""}
* module.instance_test.openstack_compute_instance_v2.instance[1]: element: element() may not be used with an empty list in:

${length(var.userdata_template) > 0 ? element(data.template_cloudinit_config.instance_userdata.*.rendered, count.index) : ""} 

[pikeas]

Looks like #53 addresses this?

[dan-rose]

We hit this problem with this evaluation,

name_prefix = "${length(var.customer_name) < 5 ? "${var.customer_name}-" : "${substr(var.customer_name, 0, 5)}-"}"

substr: 'offset + length' cannot be larger than the length of the string

[cbarbour]

length(foo) > 0 ? foo[0] : "default"

@apparentlymart: Your problem can be solved without conditional logic:

element(concat(foo, list("default"), 0)

This returns the first element of the list foo, which will be "default" if foo is empty.

More complex cases, such as @ooesili 's case can be solved using this basic pattern. His example:

resource "aws_ebs_volume" "volumes" {
  count = "${var.count}"
  snapshot_id = "${length(var.snapshot_ids) == var.count ? element(var.snapshot_ids, count.index) : ""}"
}

And the solution:

resource "aws_ebs_volume" "volumes" {
  count = "${var.count}"
  snapshot_id = "${element(concat(var.snapshot_ids, list("")), min(length(var.snapshot_ids) + 1, count.index))}"
}

With this approach, the last element of the list is an empty string. We use min() to ensure that element() will stop there rather than re-iterating through the entire list.

These solutions are entirely declarative. Arguably, they are more correct than using conditional logic.

[apparentlymart]

Thanks for sharing all the workarounds!

Please note that this is an issue about HIL itself rather than about Terraform, so (for the benefit of other readers) many of these solutions are Terraform-specific and wouldn't apply to other users of HIL unless they were to replicate some of Terraform's functions.

However, it is unlikely that this issue will ever be addressed in this codebase because Terraform is moving away from separate HIL to the new version of HCL that has HIL-like functionality integrated into it, and has a fully-functional conditional operator. That is coming in the next major version of Terraform and this particular change has its own article.

If you'd like to get updates on this being fixed in the context of Terraform, I suggest instead tracking hashicorp/terraform#15605.

If anyone monitoring this is the maintainer of another caller of the HIL library, I would suggest to investigate the new version of HCL, whose template syntax is the spiritual successor of HIL and includes the fully-featured conditional operator and a number of other enhancements.

[cbarbour]

Totally understood. My suggestions aren't intended to close the ticket, but instead to help anyone who runs into this particular problem. 👍

[MalloZup]

is this fixed with TF12?

[debugmaster]

Yes @MalloZup. Check hashicorp/terraform#15605.

[Ranger-X]

Yes @MalloZup. Check hashicorp/terraform#15605.

No, this still wont work :-(

$ terraform version
Terraform v0.12.24

$ terraform console
> false ? aws_security_group.test : null

Error: Reference to undeclared resource

  on <console-input> line 1:
  (source code not available)

A managed resource "aws_security_group" "test" has not been declared in the
> root module.

[dhoepelman]

Can confirm this is fixed

> false ? null.x : 1
1
> true ? null.x : 1
>
Error: Attempt to get attribute from null value

  on <console-input> line 1:
  (source code not available)

This value is null, so it does not have any attributes.

@Ranger-X it complains about the resource not being defined, that's a name binding issue not an evaluation issue. In most programming languages you cannot do false ? undeclared_variable : "default" either

[graywolf-at-work]

In most programming languages you cannot do

You can do that both in javascript and ruby, so it's not that unreasonable expectation.

[apparentlymart]

Hi all,

As I noted in my previous comment, Terraform no longer uses HIL as of Terraform 0.12, so this is not the right place to discuss Terraform's behaviors. If you have feedback about the behavior of the Terraform language I'd suggest opening an issue in the Terraform repository and then the Terraform team can route bug reports and enhancement requests upstream to Terraform's dependencies as appropriate.

I also noted before that this issue is unlikely to be fixed in HIL, because HIL has been superseded by the second-generation implementation of HCL now incorporating a HIL-like expression language. HIL's featureset is therefore frozen and there are no plans to change its behavior except for minor maintenance in support of any remaining applications that are still calling into it. For that reason, I'm going to close this issue as a "wont fix". If you are the maintainer of an application that is using HIL and would like conditional evaluation to behave as this issue describes, your likely path forward would be to transition to using the template language in HCL 2.