-
Notifications
You must be signed in to change notification settings - Fork 1
Data centric graph processing
Kyle Knoepfel edited this page May 11, 2023
·
4 revisions
Graph-based processing can be a very efficient and intuitive way to schedule tasks that must be executed given a set of constraints or dependencies. For example in the following graph (taken from here), the nodes represent tasks to be executed, and the edges (or arrows) indicate the flow of data required by each node to fulfill its preconditions for execution.
Although this view of graphs is very common, it does not represent the organization of data that are passed along the edges connecting the nodes. With Meld, we will "invert" the view and consider the data as nodes, and the tasks as edges that connect the data via mathematical mappings.
![](https://private-user-images.githubusercontent.com/7603566/237214232-b0c93b3c-8308-4529-90dd-bac0dc193845.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjU3NjIyMTgsIm5iZiI6MTcyNTc2MTkxOCwicGF0aCI6Ii83NjAzNTY2LzIzNzIxNDIzMi1iMGM5M2IzYy04MzA4LTQ1MjktOTBkZC1iYWMwZGMxOTM4NDUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkwOCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MDhUMDIxODM4WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NGViODFhMjAyOGIwOWUzZjM5ZjgzNzM2ZTg2MWQ3YTMyYTA5MGZlNGQyZjFmY2I3MjI0OGNiNTU1NmM1NDIzZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.lsJJe9y92WKuIhpJyNoQbI3X-6_G89bcKOzlngceI08)
The types of operations included in the above image are higher-order functions, described here.