Output error messages to a file? #2366

xiaozhour · 2022-03-18T20:16:27Z

xiaozhour
Mar 18, 2022
Collaborator

Can we output the super long message that appears on the REPL when running the model to a text file so that the actual key error message will not be overwhelmed by the other irrelevant stuff? Something like the default option in MITgcm with the STDOUT/STDERR files. For those who run on a terminal, this is probably fine because they can scroll up (assuming they don't miss it!) but in VScode there's a limit on how much you can output.

glwagner · 2022-03-19T15:28:03Z

glwagner
Mar 19, 2022
Maintainer

This is a great question, though more of a Julia issue than something specific to Oceananigans! However, there is an off-chance that there is some feature we could implement to make this easier for people.

Note that terminals also have line limits and its a problem for people working in the terminal too. I use the silly but practical solution of reducing my font size and re-executing code until l can see the top of the error message. However, there are probably other solutions.

I wrote up a little example to experiment:

using Oceananigans
grid = RectilinearGrid(size=(1, 1, 1), extent=(1, 1, 1))
model = HydrostaticFreeSurfaceModel(; grid)
simulation = Simulation(model, Δt=1.0)
bad_callback(sim) = error("Why did you write such a bad callback?")
simulation.callbacks[:error] = Callback(bad_callback)
run!(simulation)

Some hunting on google returned this thread: https://discourse.julialang.org/t/redirect-output-and-error-to-a-file/58908/3

One option is to pipe to a file with the terminal utility tee:

$ julia --project | tee out.txt

All output generated thereafter is written to out.txt. I can't really see myself using this solution though. Changing the text size is easier...

It looks it may also be possible to use the Julia module StackTraces:

https://docs.julialang.org/en/v1/manual/stacktraces/

With the above example I found

julia> err, st = try
       run!(simulation)
       catch err
       err, stacktrace()
       end
[ Info: Initializing simulation...
(ErrorException("Why did you write such a bad callback?"), Base.StackTraces.StackFrame[top-level scope at REPL[2]:4, eval at boot.jl:360 [inlined], eval_user_input(ast::Any, backend::REPL.REPLBackend) at REPL.jl:139, repl_backend_loop(backend::REPL.REPLBackend) at REPL.jl:200, start_repl_backend(backend::REPL.REPLBackend, consumer::Any) at REPL.jl:185, run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool) at REPL.jl:317, run_repl(repl::REPL.AbstractREPL, consumer::Any) at REPL.jl:305, (::Base.var"#875#877"{Bool, Bool, Bool})(REPL::Module) at client.jl:387, #invokelatest#2 at essentials.jl:708 [inlined], invokelatest at essentials.jl:706 [inlined], run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool) at client.jl:372, exec_options(opts::Base.JLOptions) at client.jl:302, _start() at client.jl:485])

julia> st
13-element Vector{Base.StackTraces.StackFrame}:
 top-level scope at REPL[2]:4
 eval at boot.jl:360 [inlined]
 eval_user_input(ast::Any, backend::REPL.REPLBackend) at REPL.jl:139
 repl_backend_loop(backend::REPL.REPLBackend) at REPL.jl:200
 start_repl_backend(backend::REPL.REPLBackend, consumer::Any) at REPL.jl:185
 run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool) at REPL.jl:317
 run_repl(repl::REPL.AbstractREPL, consumer::Any) at REPL.jl:305
 (::Base.var"#875#877"{Bool, Bool, Bool})(REPL::Module) at client.jl:387
 #invokelatest#2 at essentials.jl:708 [inlined]
 invokelatest at essentials.jl:706 [inlined]
 run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool) at client.jl:372
 exec_options(opts::Base.JLOptions) at client.jl:302
 _start() at client.jl:485julia> err, st = try
       run!(simulation)
       catch err
       err, stacktrace()
       end
[ Info: Initializing simulation...
(ErrorException("Why did you write such a bad callback?"), Base.StackTraces.StackFrame[top-level scope at REPL[2]:4, eval at boot.jl:360 [inlined], eval_user_input(ast::Any, backend::REPL.REPLBackend) at REPL.jl:139, repl_backend_loop(backend::REPL.REPLBackend) at REPL.jl:200, start_repl_backend(backend::REPL.REPLBackend, consumer::Any) at REPL.jl:185, run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool) at REPL.jl:317, run_repl(repl::REPL.AbstractREPL, consumer::Any) at REPL.jl:305, (::Base.var"#875#877"{Bool, Bool, Bool})(REPL::Module) at client.jl:387, #invokelatest#2 at essentials.jl:708 [inlined], invokelatest at essentials.jl:706 [inlined], run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool) at client.jl:372, exec_options(opts::Base.JLOptions) at client.jl:302, _start() at client.jl:485])

julia> st
13-element Vector{Base.StackTraces.StackFrame}:
 top-level scope at REPL[2]:4
 eval at boot.jl:360 [inlined]
 eval_user_input(ast::Any, backend::REPL.REPLBackend) at REPL.jl:139
 repl_backend_loop(backend::REPL.REPLBackend) at REPL.jl:200
 start_repl_backend(backend::REPL.REPLBackend, consumer::Any) at REPL.jl:185
 run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool) at REPL.jl:317
 run_repl(repl::REPL.AbstractREPL, consumer::Any) at REPL.jl:305
 (::Base.var"#875#877"{Bool, Bool, Bool})(REPL::Module) at client.jl:387
 #invokelatest#2 at essentials.jl:708 [inlined]
 invokelatest at essentials.jl:706 [inlined]
 run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool) at client.jl:372
 exec_options(opts::Base.JLOptions) at client.jl:302
 _start() at client.jl:485

I'm not sure how this can be turned into something useful, but perhaps its a starting point. It may even be possible (though probably a bit futuristic) to implement a kwarg in run!, or to provide some debugging features for users that isolate errors due to user-defined functions.

0 replies

xiaozhour · 2022-03-19T15:45:08Z

xiaozhour
Mar 19, 2022
Collaborator Author

This is helpful - thanks Greg. It's good to have some options when unable to find the error messages during debugging!

0 replies

glwagner · 2022-03-19T15:48:12Z

glwagner
Mar 19, 2022
Maintainer

Probably the best solution right now is to re-run in a terminal. But there is still some improvement to be had since that option requires re-compilation, so will slow development down.

If you find any VSCode-specific information please post here. I wonder if VSCode can pipe stdout to file?

Also we may want to convert this to a discussion, what do you think?

0 replies

glwagner · 2022-03-25T20:25:27Z

glwagner
Mar 25, 2022
Maintainer

Well, the tee solution isn't great for REPL work because

$ julia | tee out.txt

then

julia> @info "hi tim"
[ Info: hi tim

julia> error("what the heck")
ERROR: what the heck
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] top-level scope
   @ REPL[2]:1

yields the file

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.5 (2021-12-19)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

^[[?2004h^M^[[0Kjulia> ^M^[[7C^M^[[7C@info "hi tim"^M^[[0Kjulia> ^M^[[7C@info "hi tim"^M^[[21C
^[[?2004l
^M^[[0K^M^[[0Kjulia> ^M^[[7C^M^[[7C^[[?2004h^M^[[0Kjulia> ^M^[[7C^M^[[7Cerror("what the heck")^M^[[0Kjulia> ^M^[[7Cerror("what the heck")^M^[[29C
^[[?2004lERROR: what the heck
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] top-level scope
   @ REPL[2]:1

^M^[[0K^M^[[0Kjulia> ^M^[[7C^M^[[7C^[[?2004h^M^[[0Kjulia> ^M^[[7C^M^[[7C
^[[?2004l

There's probably some setting to produce simple i/o even at the REPL though.

1 reply

glwagner Mar 25, 2022
Maintainer

@TimConnelly

aramirezreyes · 2022-07-22T18:00:08Z

aramirezreyes
Jul 22, 2022

Would something like https://github.com/BioTurboNick/AbbreviatedStackTraces.jl be a good approach to abbreviate the kernelabstraction stacktraces?

0 replies

pjrusello · 2022-08-01T18:11:50Z

pjrusello
Aug 1, 2022

VS Code and most other terminals have an setting for the output buffer size. I set mine to 10000 (default is 1000) because I was getting some extra-long errors recently and couldn't see the important part of the stacktrace.

For VS Code, open File:Settings then search for "buffer" in the settings document. The specific setting is called Terminal>Integrated:Scrollback.

1 reply

glwagner Aug 7, 2022
Maintainer

Apple's iTerm2 has an option for unlimited scrollback: https://stackoverflow.com/questions/27616474/how-can-i-scroll-up-more-increase-the-scroll-buffer-in-iterm2

jagoosw · 2022-10-04T13:22:02Z

jagoosw
Oct 4, 2022
Collaborator

To get around the terminal not being long enough I use InteractiveErrors.jl which gives you a much shorter initial message and lets you copy the whole thing to the clipboard instead of printing it out. So I usually do that and then paste it into a file.

0 replies

ali-ramadhan · 2022-10-04T14:45:39Z

ali-ramadhan
Oct 4, 2022
Maintainer

For interactive debugging AbbreviatedStackTraces.jl and InteractiveErrors.jl look great.

For file output LoggingExtras.jl lets you keep terminal logging with a custom logger like OceananigansLogger plus writes the log to a file using FileLogger.

Example: https://github.com/ali-ramadhan/NeuralFreeConvection.jl/blob/7a440564a881b22a602f0a375ca5fb09b619ee92/train_free_convection_nde.jl#L114-L120

log_filepath = joinpath(output_dir, "training.log")
TeeLogger(
    OceananigansLogger(),
    MinLevelLogger(FileLogger(log_filepath), Logging.Info)
) |> global_logger

Could be used to provide stdout and stderr files.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Output error messages to a file? #2366

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 8 comments 2 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Output error messages to a file? #2366

xiaozhour Mar 18, 2022 Collaborator

Replies: 8 comments · 2 replies

glwagner Mar 19, 2022 Maintainer

xiaozhour Mar 19, 2022 Collaborator Author

glwagner Mar 19, 2022 Maintainer

glwagner Mar 25, 2022 Maintainer

glwagner Mar 25, 2022 Maintainer

aramirezreyes Jul 22, 2022

pjrusello Aug 1, 2022

glwagner Aug 7, 2022 Maintainer

jagoosw Oct 4, 2022 Collaborator

ali-ramadhan Oct 4, 2022 Maintainer

xiaozhour
Mar 18, 2022
Collaborator

Replies: 8 comments 2 replies

glwagner
Mar 19, 2022
Maintainer

xiaozhour
Mar 19, 2022
Collaborator Author

glwagner
Mar 19, 2022
Maintainer

glwagner
Mar 25, 2022
Maintainer

glwagner Mar 25, 2022
Maintainer

aramirezreyes
Jul 22, 2022

pjrusello
Aug 1, 2022

glwagner Aug 7, 2022
Maintainer

jagoosw
Oct 4, 2022
Collaborator

ali-ramadhan
Oct 4, 2022
Maintainer