koch: introduce builder #133
Conversation
This is the first implementation of #123 Builder is a new component enabling build actions to be declared as opposed to be executed imperatively. This allow for side-effect less declaration of build pipeline and enable them to be parallelized. This first implementation is incomplete and have a few issues: - Docs might be out-of-date - Cannot do bootstrapping - No CLI interface yet - Does not silence nim compiler calls (which makes it spammy in parallel runs) - Artifacts are still being placed outside of the build directory - Target description is not supported yet - koch integration is kind of hacky
| func sortTargets(b: var Builder): seq[Target] = | ||
| ## Perform a topological sort on builders' targets. | ||
| let sortedOrder = b.topologicalOrder | ||
|
|
||
| var | ||
| newNames: typeof(b.names) | ||
| newRecipes: typeof(b.recipes) | ||
| newDependencies: typeof(b.dependencies) | ||
|
|
||
| for idx, target in sortedOrder.pairs: | ||
| # Insert to new target seq according to the sorted order | ||
| newRecipes.add b.recipes[target] | ||
| newNames.add b.names[target] | ||
|
|
||
| # If the target has any dependencies | ||
| let deps = b.dependencies.getOrDefault(target) | ||
| if deps != {}: | ||
| for newDep in Target(0) ..< target: | ||
| let dep = sortedOrder[newDep] | ||
| # If dep was a dependency of the target | ||
| if dep in deps: | ||
| # Re-link it to the new ID | ||
| newDependencies[target].incl newDep | ||
|
|
||
| b.names = newNames | ||
| b.recipes = newRecipes | ||
| b.dependencies = newDependencies |
| a: Operand ## First operand | ||
| b: Operand ## Second operand | ||
|
|
||
| Builder* = object |
There was a problem hiding this comment.
I would lead with this, since type sections rearrange -- this gives a nice map of the concepts/data.
| import std/[os, osproc, parseopt, sequtils, setutils, tables] | ||
|
|
||
| type | ||
| Target* = distinct uint16 |
There was a problem hiding this comment.
General remark for DOD: Part of me would consider reserving Target and such names for 'view objects' that provide an API over the internals; but the builder isn't really a library right now so I wouldn't worry about it yet.
| b.recipes = newRecipes | ||
| b.dependencies = newDependencies | ||
|
|
||
| func name*(b: Builder, target: Target): string = |
There was a problem hiding this comment.
This is where it might be nice to have Target be TargetId and Target be:
type
Target = object
builder: Builder
id: TargetIdat least from the outside people are dealing with a friendlier concept of a holistic target, while internally the memory layout and data are nice and orderly.
There was a problem hiding this comment.
It will be at one point, but having builder as a ref is not something I'd like now due to how easy it is to hide mutations.
There was a problem hiding this comment.
Well it could only happen inside module defined procs and if you wanted extra safety the builder could be a distinct to reduce the surface further.
| @@ -243,6 +240,15 @@ proc buildTools(args: string = "") = | |||
| "--opt:speed --stacktrace -d:debug --stacktraceMsgs -d:nimCompilerStacktraceHints --excessiveStackTrace:off " & defineSourceMetadata() & " " & args, | |||
| outputName = "nim_dbg") | |||
|
|
|||
| proc addTools(b: var Builder) = | |||
| let tools = [ | |||
There was a problem hiding this comment.
maybe a b.buildGroup call that takes a name and a bunch of addxxx calls, returning the name as an alias?
| let compilerManifest = nimSource / "compiler" / "installer.ini" | ||
|
|
||
| let buildScript: Target = b.addRun("installscripts", niminst, @defaultArgs & @["scripts", compilerManifest]) | ||
| discard b.addRun("archive", niminst, @defaultArgs & @["--format:tar.xz", "archive", compilerManifest], dependsOn = {buildScript}) |
There was a problem hiding this comment.
few cases where I think discardable makes sense, I don't think it catches errors here; because the dependency passing will force that.
Co-authored-by: Saem Ghani <[email protected]>
| ## Returns the path to the nimcache of the executable | ||
| b.buildDir / "nimcache" / b.name(exe) | ||
|
|
||
| func addRun*(b: var Builder, name: string, exe: Executable, |
There was a problem hiding this comment.
To get transactional behavior:
- Immediately make a copy of b
- Wrap the current body in a try
- manipulate the copy instead of b in the try
- in the last line of the try copy back the changes over from the copy into b
- profit ... and by that I mean you probably want to make that a template
This is the first implementation of #123
Builder is a new component enabling build actions to be declared as
opposed to be executed imperatively. This allow for side-effect less
declaration of build pipeline and enable them to be parallelized.
This library is heavily influenced by Meson with a touch of zig build.
This first implementation is incomplete and have a few issues, posted here so everyone can keep track and make suggestions:
runs)
cleanfunctioninstallfunction