Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More efficient implementation of ignore_above in synthetic source for flattened fields #110239

Open
lkts opened this issue Jun 27, 2024 · 1 comment
Labels

Comments

@lkts
Copy link
Contributor

lkts commented Jun 27, 2024

Description

#110214 handles synthetic source for flattened fields when ignore_above is used (child fields ignored due to ignore_above are now present in synthetic source). In that PR if there is any ignored child fields encountered during parsing it stores the entire object in _ignored_source field. This can be optimized by storing only ignored child fields in dedicated stored fields and then merging such values with valid child fields read from doc values. This is similar to how ignore_malformed works with synthetic source for other field types. This optimization will reduce storage space needed to handle this case.

@lkts lkts added >enhancement needs:triage Requires assignment of a team area label :StorageEngine/Mapping The storage related side of mappings labels Jun 27, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants