Closed
Description
Describe the bug
When a test suite doesn't contain contexts, the default ordering of tests is determined by test procedure position in package specification.
When test suite contains a context, the ordering is no longer directly related to test or context position in test suite.
Provide version info
Affects all versions that support contexts.
To Reproduce
Create two test packages. One with contexts one without.
create or replace package demo_1 as
--%suite
--%test(First)
procedure t1;
--%test(Second)
procedure t2;
--%test(Third)
procedure t3;
--%test(Forth)
procedure t4;
end;
/
create or replace package body demo_1 as
procedure t1 is begin null; end;
procedure t2 is begin null; end;
procedure t3 is begin null; end;
procedure t4 is begin null; end;
end;
/
create or replace package demo_2 as
--%suite
--%test(First)
procedure t1;
--%context(Context 1)
--%test(Second)
procedure t2;
--%endcontext
--%context(Context 2)
--%test(Third)
procedure t3;
--%endcontext
--%test(Forth)
procedure t4;
end;
/
create or replace package body demo_2 as
procedure t1 is begin null; end;
procedure t2 is begin null; end;
procedure t3 is begin null; end;
procedure t4 is begin null; end;
end;
/Run tests:
call ut.run('demo_1,demo_2');
Results are:
demo_2
Context 2
Third [.006 sec]
Context 1
Second [.003 sec]
First [.003 sec]
Forth [.003 sec]
demo_1
First [.001 sec]
Second [.001 sec]
Third [.001 sec]
Forth [.001 sec]
Finished in .028732 seconds
8 tests, 0 failed, 0 errored, 0 disabled, 0 warning(s)
Expected behavior
Run tests:
call ut.run('demo_1,demo_2');
Results are:
demo_2
First [.003 sec]
Context 1
Second [.003 sec]
Context 2
Third [.006 sec]
Forth [.003 sec]
demo_1
First [.001 sec]
Second [.001 sec]
Third [.001 sec]
Forth [.001 sec]
Finished in .028732 seconds
8 tests, 0 failed, 0 errored, 0 disabled, 0 warning(s)
Additional context
The ordering of tests is mostly relevant when trying to write tests as documentation for your code, so that the tests tell a story.