Improve logging when skipping values or entities in mappings

[tillprochaska]

This is a follow-up to #1404. This PR only includes the changes that improve logging as I agree that completely disabling value cleaning can be dangerous.

[tillprochaska]

@pudo You suggested also logging the proxy ID here. I can see how this can be useful, especially when the mapping uses id_column. I just don’t know how to implement this in a smart way as the proxy ID hasn’t been computed yet at this point (and it cannot be computed earlier as that requires knowing the property values).

Only option I see is not to log the warning message directly, but cache them until the ID has been computed, something like this:

discarded_values: dict[Property, set[value]] = defaultdict(set)

for prop in self.properties:
  if prop.prop.type == registry:country:
    # Save discarded values
    discarded_values = prop.map(proxy, record, entities)
    discarded_values[prop].update(discarded_values)

for prop in self.properties:
  if prop.prop.type == registry:country:
    # Save discarded values
    discarded_values = prop.map(proxy, record, entities)
    discarded_values[prop].update(discarded_values)

proxy.id = self.compute_key(record)
if proxy.id is None:
  if self.id_column:
    log.warn(...)
  if self.keys:
    log.warn(...)
  return None

if discarded_values:
  # Finally about discarded values here
  for prop, values in discarded_values.items():
    log.warn(...)

But that’s a lot of code and indirection for a simple log message… Any good ideas?

[pudo]

Yeah I think it's too much faff in the core loop of the mapper. Let's leave it.

[pudo]

Would it be OK to merge this and make a minor version release?

[pudo]

Yeah I think it's too much faff in the core loop of the mapper. Let's leave it.

[tillprochaska]

@pudo Thanks for taking another look, just merged the PR! :)