[IR] Support symbolic values?

[justinchuby]

Create a field in Value to support representing equivalence relations among values.

1. Value' is an alias of Value, e.g. output of identity or +0
2. Value is a Sequence containing known values as elements

[justinchuby]

Before this is created, I recommend using a key in the meta dictionary to store this information. It is possible to create a helper function to interact with the dictionary. Replacing it with the new field when the field is implemented should be easy.

Another question is how we can make the field robust. Is there a lightweight pass we can run every time the graph is changed? Do we need to create any invariance on this field?

[justinchuby]

cc @gramalingam

I came up with a new idea for this. It may be helpful to create an auxiliary data structure for this, rather than making it a field in values since it is more usage specific. Consider a data structure with these apis:

1. mark_equivalence(value_1, value_2) and mark_equivalence(value_1, [<sequence of values>]) Establishes that two values are equivalent
2. unmark_equivalence(value_1, value_2) breaks the equivalence relationship
3. assign(value, ir.Tensor) binds a constant tensor to a value, and internally also binds it to all values that share the equivalence relation
4. are_equivalent(value_1, value_2) returns whether two values are the same
5. etc.

This way it is very easy to establish and manage equivalence relationships. Internally it can be represented as a tree like structure.

[gramalingam]

Keeping it a separate data-structure makes sense: making it clear it is the user's responsibility in all regards. Agree with that. However, a dictionary mapping an IR value to its symbolic value would work better for this particular usage than an equivalence-relation data-structure. I think I will try that.