What if we just wanna run one test file?

[navidcy]

At the moment, supposedly we can change

Oceananigans.jl/test/dependencies_for_runtests.jl

Line 82 in 3bb62a6

test_file = get(ENV, "TEST_FILE", :none) |> Symbol

to

test_file = "test_some_new_feature.jl"

and then

Oceananigans.jl/test/runtests.jl

Lines 6 to 10 in 3bb62a6

	if test_file != :none
	@testset "Single file test" begin
	include(String(test_file))
	end
	end

will run that test file. But then, the runtest.jl will continue running the whole test suite!

We'd like to stop after testing the test_file, right?

cc @josuemtzmo

[navidcy]

This should do the job I believe

@testset "Oceananigans" begin
    if test_file != :none
        @testset "Single file test" begin
            include(String(test_file))
        end
    else
        # Core Oceananigans
        if group == :unit || group == :all
              .....
        end
    end
end

[glwagner]

I think the best way is to go to the REPL and write

julia> include("one_test_file.jl")

We've designed all the test files to work that way.

[navidcy]

Well... Some packages that the test use are not installed and you need to install manually and then be careful not to commit the Manifest. I get annoyed all the time.

But in any case: the test_file is wrong because it tests the file PLUS everything else.

[glwagner]

Sorry, can you fix it then?

[navidcy]

oh no need to apologize!
I was only wondering if I'm missing something here of if there was a different intented functionality.

I'll open a PR

[glwagner]

I put a ton of effort into making it so that we could include one file and run the tests. So if its broken or wasn't a complete solution hopefully we can fix it, or make it work more generally. It's more annoying to have to include the filename as a global variable. I don't like that workflow as much. When we can include a file, we can rapidly iterate because we don't have to wait for things to recompile.

[navidcy]

You are right! Being able to do

julia> include("this_test_script.jl")

and bit later

julia> include("this_other_test_script.jl")

is really helpful.

However, there are a few packages that are different in the test environment (e.g., test environment uses DataDeps and TimesDates). Thus, often one needs to first install those packages before they are able to include("this_other_test_script.jl").

The latest changes suggested in #3511 don't break this current functionality but also allow for one to run a single test file as a suite using the proper testing environment via:

$ TEST_FILE=this_test_script.jl julia --project -e"using Pkg; Pkg.test()"

[glwagner]

However, there are a few packages that are different in the test environment (e.g., test environment uses DataDeps and TimesDates). Thus, often one needs to first install those packages before they are able to include("this_other_test_script.jl").

If you find yourself running lots of tests then its best to put these packages in your global environment. Then this pattern will work.

[josuemtzmo]

That's a good point. As a newcomer it will be nice to have some documentation of this, since otherwise, it is hard to know how to test specific files in the test suit.

[navidcy]

We should include a bit more information about testing and suggestions for a convenient workflow at https://clima.github.io/OceananigansDocumentation/stable/contributing/

[glwagner]

For sure. The trick about the global environment is just Julia knowledge. But it is relevant to efficient workflows for testing so we can indeed put it there.

Personally I don't test just one file unless I am debugging major refactors but perhaps others do it differently...