-
|
Hello, One possibility might be to use Also, how to do all that in the same terminal instead of multiple ones? Best, |
Beta Was this translation helpful? Give feedback.
Replies: 15 comments 17 replies
-
Do you really need
Same: technically, you do not need an intermediate file (but feel free to generate one anyway if that makes sense in your workflow).
I see two main approaches: Approach 2: as you suggest, having markers in the output of ... and turns each title into a Moulti step. That way, Moulti users could "simply" reformat the output of their existing scripts to integrate them with Moulti. This is essentially a less specialized variant of what
Although the quickstart makes you play with two terminals at first, the documentation also details |
Beta Was this translation helpful? Give feedback.
-
I would very much like to avoid doing so, I am happy to read that you think it can be avoided altogether.
This is what I undertood I should do reading the Documentation.md, and it seemed unrealistic to create a whole new makefile system to accomodate for moulti.
See answer for Approach1.
That would be ideal! the moulti instructions could even be hidden behind a "erase current line" terminal command so that it is not visible when running on a system where moulti is not installed. I had a great experience with the klogg app https://klogg.filimonov.dev/ where you can open any log, then search for things and matched lines would list in a window bellow. You could also define regexes and styles to be applied (fg / bg color, bold etc). I guess one direction moulti development could take is follow that approach where a user create some configuration file that corresponds to the way its existing scripts already work, and then wrap the launch of such script. For example, launching make with Here the steps are started by "Must remake target" and completed by "Successfully remade target", where "[X/Y]" is the progress of such target. If moulti has some DSL to express that in the command line, or with some ad-hoc config file ,it would be easier to integrate with existing machinery. Or I could create a filter that calls moulti appropriately when such strings appear, but I am not even sure how to do that. |
Beta Was this translation helpful? Give feedback.
-
Like this?
#!/usr/bin/env bash
# Instruct the Moulti instance to harvest any output not explicitly assigned to
# a step:
export MOULTI_RUN_OUTPUT=harvest
export MOULTI_INSTANCE="make"
[ "${MOULTI_RUN}" ] || exec moulti run -- "$0" "$@"
function process_lines {
local step_counter=0
while read -r line; do
if [[ "${line}" =~ 'Must remake target' ]]; then
# New target: create a new Moulti step:
local title=$(sed 's/Must remake target `\(.*\)'\''./\1/' <<< "${line}")
((++step_counter))
local step_id="make_$$_${step_counter}"
moulti step add "${step_id}" --title="${title}" --bottom-text=' ' < /dev/null
# Redirect stdout to `moulti pass our_new_step`
exec > >(moulti pass "${step_id}")
fi
[[ "${line}" =~ 'Successfully remade target' ]] && moulti step update "${step_id}" --classes=success
[[ "${line}" =~ error ]] && moulti step update "${step_id}" --classes=error
printf '%s\n' "${line}"
done
}
# Instance title:
moulti set --title="$*"
# Run the command given on the command-line and process each line of its
# output:
"$@" 2>&1 | process_linesUsage: chmod +x moultimake.bash
./moultimake.bash make -dRegarding the rest of your suggestions: thank you, I will take all of this into account when designing the next features. |
Beta Was this translation helpful? Give feedback.
-
|
It works!!! That’s so cool! I think that example that you just created can easily be adapted to other "filter" use cases, and should be put in the documentation somewhere close to the beginning. :) |
Beta Was this translation helpful? Give feedback.
-
Glad you like it.
It will definitely serve as a basis for my next reflections. Specifically:
|
Beta Was this translation helpful? Give feedback.
-
Yes! And it was easy now with this template to complete with some progress bar \o/ see #!/usr/bin/env bash
# Instruct the Moulti instance to harvest any output not explicitly assigned to
# a step:
export MOULTI_RUN_OUTPUT=harvest
export MOULTI_INSTANCE="make"
[ "${MOULTI_RUN}" ] || exec moulti run -- "$0" "$@"
function process_lines {
local step_counter=0
while read -r line; do
if [[ "${line}" =~ 'Must remake target' ]]; then
# New target: create a new Moulti step:
local title=$(sed 's/Must remake target `\(.*\)'\''./\1/' <<<"${line}")
((++step_counter))
local step_id="make_$$_${step_counter}"
moulti step add "${step_id}" --title="${title}" --bottom-text=' ' </dev/null
# Redirect stdout to `moulti pass our_new_step`
exec > >(moulti pass "${step_id}")
fi
if echo ${line} | grep -q '^\[[0-9]*\/[0-9]*\]'; then
local progress_status=$(echo ${line} | sed -e 's/^\[\([0-9]*\)\/\([0-9]*\).*/\1/')
local progress_target=$(echo ${line} | sed -e 's/^\[\([0-9]*\)\/\([0-9]*\).*/\2/')
moulti set --progress-bar --progress="${progress_status}" --progress-target="${progress_target}"
fi
[[ "${line}" =~ 'Successfully remade target' ]] && moulti step update "${step_id}" --classes=success && moulti set --no-progress-bar
[[ "${line}" =~ error ]] && moulti step update "${step_id}" --classes=error
printf '%s\n' "${line}"
done
}
# Instance title:
moulti set --title="$*"
# Run the command given on the command-line and process each line of its
# output:
"$@" 2>&1 | process_linesRight now, when some steps have happened, I cannot know what happens or happened until I scroll with the mouse over moulti. klogg has a While moulti already scrolls inside steps, could moulti also have something similar to scroll finished/inactive steps ? (should I open new issues?)
That would be my case I guess.
When I see how concise this moultimake.bash is, I wonder if it is necessary at all. Do you have any idea how a declarative file would look like? |
Beta Was this translation helpful? Give feedback.
-
Moulti has programmatic scrolling too. Refer to the documentation for details but TL;DR:
|
Beta Was this translation helpful? Give feedback.
-
That will be part of my reflections too.
The default approach would be a serializable Python dict written as JSON/YAML/TOML that describes:
+ support for using regex captures and environment variables in actions. I could implement that by myself OR I could look for a tool/library/framework that already provides this kind of logic. |
Beta Was this translation helpful? Give feedback.
-
That’s very neat! I wonder how I missed that when searching in the documentation. Thanks!
I tried those, but I have troubles understanding if scroll stops, or if scripts takes time. I will use
Reading all of this, I wonder if a python API might be easier to implement (and more powerful) than JSON/YAML/TMOL. This api would read like declarative, but really just call python functions, a bit like the cmake syntax. import moulti as m
m.detector.add_new_step_rule("Must remake target `\(.*\)'\'", auto_increment_id=True, show_progress_bar=False)
m.detector.add_step_success_rule(".*Successfully remade target.*")
m.detector.add_step_error_rule(".*error.*")
m.detector.add_step_status_progress_rule("^\[\([0-9]*\)\/[0-9]*.*", force_show_progress_bar=True)
m.detector.add_step_total_progress_rule("^\[[0-9]*\/\([0-9]*\).*", force_show_progress_bar=True)
m.add_action(["make", "-d", "configure"], can_be_restarted=True, on_success="next_command")
m.add_action(["make", "-d", "build"], can_be_restarted=True, on_success="next_command")
m.add_action(["make", "-d", "test"], can_be_restarted=True)
m.run(run_output="harvest", instance="make", start_instance_if_needed=True, follow=True)Later part can easily be made customizable using vanilla python: for i in range(1,len(argv)):
m.add_restartable_command(["make", "-d", argv[i)], can_be_restarted=True, on_success="next_command")
m.run(run_output="harvest", instance="make", start_instance_if_needed=True, follow=True) |
Beta Was this translation helpful? Give feedback.
-
They are complementary features: |
Beta Was this translation helpful? Give feedback.
-
It is planned to implement a "Python User API", i.e. a Python API exposed to users and that would adhere to semver. It will definitely be more powerful (e.g. it should integrate a class that keeps track of all created steps, and it should not need as many fork-exec-connect-send-receive-close-exit operations as a shell script) but this will likely wait until things get more stable (because a stable feature set makes it easier to design a perennial API). My reflections so far:
By the way, real life use cases are most welcome to feed those reflections. Do you have any other use case in mind? |
Beta Was this translation helpful? Give feedback.
-
I find it hard to achieve the perfect API first try. You can also open a beta python user API which adhere to semver by starting at major version
I find it easier to launch moulti once, when it is done maybe inspect what I need, then close and continue working. This workflow reduces the number of terminals I need to keep open. Maybe there is another way to work, which is to have one moulti session, and throw at it work to do from multiple context (I use multiple git worktrees sometimes). But then, I would not know how to interpret the result in case of failure: which context did fail?
Sure, that’s reasonable.
Definitely! Focusing on real-case solution, maybe it will become more obvious which small details or small options are missing that could make such functions easier to write.
Finite-State Machine could be avoided here by keeping the last 3 lines (instead of the last 1 line) when deciding if this is a new step:
The makefile was a real use case. Another possible use case is tailing some build job in a distributed CI system: right now, I can do a "while not finished, Another possible use case is podcast downloading: I select podcast episode from another software, then spawn one terminal per podcast episode to download, wait for the downloads, then clean the finished terminals. I could use moulti to have some queue, where I could append download jobs to it. However, it is not really useful, as all downloads eventually complete, and I can just hide the downloading terminals away while multitasking. What are the real use case that you imagined ? I mean, apart from Ansible recipes. |
Beta Was this translation helpful? Give feedback.
-
Indeed. That should be feasible. Feeding multiple steps in parallel is demonstrated in
If the downloading order is purely sequential, Moulti should do. However, if there are multiple parallel downloads, Moulti may fall short since it offers only one global progress bar.
My main use case has always been scripts with well-defined but numerous steps and a varying amount of output (from no output at all to Overall, Moulti ought to be relevant as long as these conditions are met:
|
Beta Was this translation helpful? Give feedback.
-
|
Toying with bash: moulti/examples/moulti-functions.bash Lines 74 to 115 in 2556c4e Rewriting #!/usr/bin/env bash
# Instruct the Moulti instance to harvest any output not explicitly assigned to
# a step:
export MOULTI_RUN_OUTPUT=harvest
export MOULTI_INSTANCE="make"
[ "${MOULTI_RUN}" ] || exec moulti run -- "$0" "$@"
source moulti-functions.bash
MOULTI_NEW_STEP_PATTERN='Must remake target `(.*).\.'
function moulti_make_step {
moulti step add "$2" --title="$5" --bottom-text=' ' --scroll-on-activity=-1 && echo "$2"
}
function moulti_inspect_line {
[[ "${1}" =~ 'Successfully remade target' ]] && \
moulti step update "${2}" --classes=success && moulti set --no-progress-bar
[[ "${1}" =~ error ]] && moulti step update "${2}" --classes=error
if [[ "${1}" =~ \[([0-9]+)/([0-9]+)\] ]]; then # e.g. "[19/20]"
moulti set --progress-bar --progress="${BASH_REMATCH[1]}" --progress-target="${BASH_REMATCH[2]}"
fi
[[ "$1" =~ ${MOULTI_NEW_STEP_PATTERN} ]] || echo "${line}"
}
# Instance title:
moulti set --title="$*"
# Run the command given on the command-line and process each line of its
# output:
"$@" 2>&1 | moulti_process_lines
I do not think it covers all possible corner cases, but that seems like a practical approach. What do you think? |
Beta Was this translation helpful? Give feedback.
-
Ok, I have found how to do that. The only downside is that |
Beta Was this translation helpful? Give feedback.
-
|
The change was made in the |
Beta Was this translation helpful? Give feedback.
-
I can confirm now that One step less to install moulti and get ready to try examples. Also, makes the bash fonctions discoverable because it can be completed. |
Beta Was this translation helpful? Give feedback.
-
|
And this is now part of Moulti 1.9.0! |
Beta Was this translation helpful? Give feedback.
-
Done in the devel branch: https://github.com/xavierog/moulti/tree/devel/examples |
Beta Was this translation helpful? Give feedback.
-
|
Would it be possible to collapsed finished green steps automatically ? that would reduce clutter even more. |
Beta Was this translation helpful? Give feedback.
-
|
Sure. In moulti step update "${2}" --classes=success && moulti set --no-progress-barwith: moulti step update "${2}" --classes=success --collapsed && moulti set --no-progress-barIf you want to insert a 5-second delay between the step turning green and the step being collapsed, use this instead: moulti step update "${2}" --classes=success && moulti set --no-progress-bar && { moulti_delayed_collapse "${2}" 5 & } |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Not much to add apart from the fact that I use your script daily :) thanks for creating it! |
Beta Was this translation helpful? Give feedback.
-
|
Glad to read that!
|
Beta Was this translation helpful? Give feedback.
Like this?
moultimake.bash: