-
Notifications
You must be signed in to change notification settings - Fork 223
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added low-level mechanism to run weaver apps. #651
Open
mwhittaker
wants to merge
1
commit into
main
Choose a base branch
from
run
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR introduces a new low-level mechanism, `run.Run`, to run a weavelet when the topology of the application (e.g., the address of every listener and component) is static and predetermined. To draw an analogy, git distinguishes between [plumbing and porcelain][git]. Plumbing commands are complicated low-level commands. Porcelain commands are simple, high-level commands that are built on top of plumbing commands. Most users use porcelain commands most (or all) of the time, but plumbing commands exist to make more complex git operations possible (though not necessarily convenient). For Service Weaver, `weaver single`, `weaver multi`, `weaver ssh`, `weaver kube`, and `weaver gke` are porcelain commands. We don't offer any plumbing commands, besides writing a deployer completely from scratch. This PR aims to introduce a generally useful plumbing command. > Motivation Here's our vision for how this PR will be useful. The details are not final, but I want to share something concrete to motivate the PR. We'll add a user-facing plumbing command on top of the `run` package, let's say `weaver run`. `weaver run` will take a topology file describing the topology of the application, say `topo.toml`: ```toml deployment_id = "v1" topology = { "Main": ["localhost:9000"], "Odd": ["localhost:9001"], "Even": ["localhost:9002"], } groups = [ { name: "main", components: ["Main"], listeners: {"collatz": "localhost:8000"}, address: "localhost:9000", }, { name: "odd", components: ["Odd"], address: "localhost:9001", }, { name: "even", components: ["Even"], address: "localhost:9002", }, ] ``` Then, you can run an application manually like this: ```shell $ weaver run weaver.toml topo.toml odd # run "odd" group $ weaver run weaver.toml topo.toml even # run "even" group $ weaver run weaver.toml topo.toml main # run "main" group ``` This command will be generally useful, but our motivating use case is to enable people to run Service Weaver applications on Kubernetes with a level of control not possible with `weaver kube`. [git]: https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain
mwhittaker
added a commit
that referenced
this pull request
Oct 5, 2023
```console $ weaver exec --help Execute a Service Weaver application. Usage: weaver exec <config file> <topology file> <node> Flags: -h, --help Print this help message. Description: "weaver exec" is low-level command to run a Service Weaver application when the topology of the application (e.g., the address of every listener and component) is static and predetermined. It is more convenient to deploy an application using a deployer like "weaver multi" or "weaver kube". "weaver exec" gives you more control over how you deploy your application, but this comes at the cost of being more cumbersome to use. Topology Files: To deploy an application using "weaver exec", you must specify the topology of your application in a TOML file. For example, consider the following topology file for a Service Weaver application with components Main, Foo, and Bar where the Main component exports a listener named lis. ┌────────────────────topology.toml──────────────────────┐ │ id = "v1" │ │ │ │ [nodes.main] │ │ address = "localhost:9000" │ │ components = ["github.com/ServiceWeaver/weaver/Main"] │ │ listeners.lis = "localhost:8000" │ │ │ │ [nodes.foobar] │ │ address = ":9001" │ │ dial_address = "localhost:9001" │ │ components = [ │ │ "github.com/example/app/Foo", │ │ "github.com/example/app/Bar", │ │ ] │ └───────────────────────────────────────────────────────┘ A topology file begins with a unique deployment id (e.g., "v1"). Next is a series of node definitions. Every node has a name (e.g., "main", "foobar") and the following fields: - address: The address of the node. - dial_address: The dialable address of the node (defaults to address). - components: A list of the components hosted on the node. - listeners: A map from listener names to listener addresses. In the example above, - The "main" node runs on address "localhost:9000" with dialable address "localhost:9000". It hosts the Main component and exports listener "lis" on "localhost:8000". - The "foobar" node listens on address ":9001" with dialable address "localhost:9001". It hosts components Foo and Bar. Note that nodes are a logical concept. You can run multiple nodes on the same physical machine, or you can run multiple nodes on different machines. To deploy an application with "weaver exec", you provide the weaver config file, the topology file, and the name of the node to execute. To run the example above, we would run the following two commands: $ weaver exec weaver.toml topology.toml foobar # run the foobar node $ weaver exec weaver.toml topology.toml main # run the main node ``` See #651 for context.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a new low-level mechanism,
run.Run
, to run a weavelet when the topology of the application (e.g., the address of every listener and component) is static and predetermined.To draw an analogy, git distinguishes between plumbing and porcelain. Plumbing commands are complicated low-level commands. Porcelain commands are simple, high-level commands that are built on top of plumbing commands. Most users use porcelain commands most (or all) of the time, but plumbing commands exist to make more complex git operations possible (though not necessarily convenient).
For Service Weaver,
weaver single
,weaver multi
,weaver ssh
,weaver kube
, andweaver gke
are porcelain commands. We don't offer any plumbing commands, besides writing a deployer completely from scratch. This PR aims to introduce a generally useful plumbing command.Motivation
Here's our vision for how this PR will be useful. The details are not final, but I want to share something concrete to motivate the PR. We'll add a user-facing plumbing command on top of the
run
package, let's sayweaver run
.weaver run
will take a topology file describing the topology of the application, saytopo.toml
:Then, you can run an application manually like this:
This command will be generally useful, but our motivating use case is to enable people to run Service Weaver applications on Kubernetes with a level of control not possible with
weaver kube
.