Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Feature/1042 add and to tags #1250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 35 commits into from
Apr 28, 2023
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
0daab33
Checkpoint
lwasylow Mar 28, 2023
adbc76e
Address too long identified in 11g.
lwasylow Mar 28, 2023
1478b0d
Adding validation for tag expression
lwasylow Mar 29, 2023
b5ad747
Comment out to see why its failing.
lwasylow Mar 29, 2023
06cb054
Revert "Comment out to see why its failing."
lwasylow Mar 29, 2023
e87d39f
Adding validate function, with no calls
lwasylow Mar 29, 2023
97537de
Remove a & from text
lwasylow Mar 29, 2023
2a0f99a
Extra changes and added tests
lwasylow Mar 31, 2023
5b46140
Merge branch 'develop' of https://github.com/utPLSQL/utPLSQL into fea…
lwasylow Mar 31, 2023
b30688c
Address sonar coverage issues.
lwasylow Apr 1, 2023
0c41a0f
Adding tests covering exception of invalid tags
lwasylow Apr 1, 2023
543685d
Removing that , we will not implement that, there is no benefit at th…
lwasylow Apr 1, 2023
0d3cfa1
Removing force
lwasylow Apr 1, 2023
20e3177
Changing to use Dijkstra algorithm to parse infix notation into postf…
lwasylow Apr 10, 2023
f51cc99
Missing slash at end of type
lwasylow Apr 10, 2023
4b8e2ab
Cleanup.
lwasylow Apr 10, 2023
84e8684
Update tests after removed function
lwasylow Apr 10, 2023
2e7a766
Tidy up tests
lwasylow Apr 10, 2023
cbdf83a
Added ut_stack to uninstall
lwasylow Apr 10, 2023
436eb5b
Addressing test failures and sonar smells
lwasylow Apr 11, 2023
3d77514
Update name
lwasylow Apr 11, 2023
bf6959f
Update tests and code
lwasylow Apr 11, 2023
d8233ff
fixing typo in docs
lwasylow Apr 12, 2023
bd860f6
Removed unused variable
lwasylow Apr 12, 2023
313d5e9
Stage 1 Resolving PR comments
lwasylow Apr 13, 2023
02a071c
Separate tag logic.
lwasylow Apr 13, 2023
b8b66ee
Fix uninstall
lwasylow Apr 14, 2023
077fdb1
Various PR fixe
lwasylow Apr 14, 2023
01e5364
Update tests and code
lwasylow Apr 15, 2023
dc0b4a6
Addressing changes via PR review.
lwasylow Apr 18, 2023
ef1c02b
Update docs
lwasylow Apr 18, 2023
1551ea5
Adding any and none
lwasylow Apr 25, 2023
9dee7e0
Update docs
lwasylow Apr 26, 2023
beb9a3a
Resolving PR
lwasylow Apr 27, 2023
46ffe73
Update note
lwasylow Apr 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 10 additions & 78 deletions 88 docs/userguide/annotations.md
Original file line number Diff line number Diff line change
Expand Up @@ -1616,93 +1616,25 @@ or

Tags are defined as a comma separated list within the `--%tags` annotation.

When executing a test run with tag filter applied, the framework will find all tests associated with the given tags and execute them.
The framework applies `OR` logic to all specified tags so any test / suite that matches at least one tag will be included in the test run.

When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent. Parent suite tests are not executed, but a suitepath hierarchy is kept.


Sample test suite package with tags.
```sql linenums="1"
create or replace package ut_sample_test is

--%suite(Sample Test Suite)
--%tags(api)

--%test(Compare Ref Cursors)
--%tags(complex,fast)
procedure ut_refcursors1;

--%test(Run equality test)
--%tags(simple,fast)
procedure ut_test;

end ut_sample_test;
/

create or replace package body ut_sample_test is

procedure ut_refcursors1 is
v_actual sys_refcursor;
v_expected sys_refcursor;
begin
open v_expected for select 1 as test from dual;
open v_actual for select 2 as test from dual;

ut.expect(v_actual).to_equal(v_expected);
end;

procedure ut_test is
begin
ut.expect(1).to_equal(0);
end;

end ut_sample_test;
/
```

Execution of the test is done by using the parameter `a_tags`

```sql linenums="1"
select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api'));
```
The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api`

```sql linenums="1"
select * from table(ut.run(a_tags => 'complex'));
```
The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex`

```sql linenums="1"
select * from table(ut.run(a_tags => 'fast'));
```
The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_test.ut_test` tests, as both tests are tagged with `fast`
When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent, unless they are excluded explicitly at runtime with a negated tag expression.
lwasylow marked this conversation as resolved.
Show resolved Hide resolved
See [running unit tests](running-unit-tests.md) for more information on using tags to filter test suites that are to be executed.

#### Tag naming convention

Tags must follow the below naming convention:

- tag is case sensitive
- tag can contain special characters like `$#/\?-!` etc.
- tag cannot be an empty string
- tag must not contain any of the following reserved characters:
- comma (,)
- left or right parenthesis ((, ))
- ampersand (&)
- vertical bar (|)
- exclamation point (!)
- tag cannot be null or blank
- tag cannot start with a dash, e.g. `-some-stuff` is **not** a valid tag
- tag cannot contain spaces, e.g. `test of batch`. To create a multi-word tag use underscores or dashes, e.g. `test_of_batch`, `test-of-batch`
lwasylow marked this conversation as resolved.
Show resolved Hide resolved
- leading and trailing spaces are ignored in tag name, e.g. `--%tags( tag1 , tag2 )` becomes `tag1` and `tag2` tag names


#### Excluding tests/suites by tags

It is possible to exclude parts of test suites with tags.
In order to do so, prefix the tag name to exclude with a `-` (dash) sign when invoking the test run.

Examples (based on above sample test suite)

```sql linenums="1"
select * from table(ut.run(a_tags => 'api,fast,-complex'));
```
The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`.
Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed.

- tag cannot be one of two reserved words : `none` and `any`, any tags with that will not be considered.
lwasylow marked this conversation as resolved.
Show resolved Hide resolved


### Suitepath
Expand Down
203 changes: 196 additions & 7 deletions 203 docs/userguide/running-unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,23 +319,212 @@ select * from table(ut.run('hr.test_apply_bonus', a_random_test_order_seed => 30

In addition to the path, you can filter the tests to be run by specifying tags. Tags are defined in the test / context / suite with the `--%tags`-annotation ([Read more](annotations.md#tags)).
Multiple tags are separated by comma.
The framework applies `OR` logic to all specified tags so any test / suite that matches at least one tag will be included in the test run.


### Tag Expressions

Tag expressions are boolean expressions created by combining tags with the `!`, `&`, `|` operators. Tag expressions can be grouped using `(` and `)` braces. Grouping tag expressions affects operator precedence.

Two special expressions are supported, `any` and `none`, which select all tests with any tags at all, and all tests without any tags, respectively. These special expressions may be combined with other expressions just like normal tags. When using `none` be aware that if the suite is tagged it will exclude any tests and children belonging to that suite.
lwasylow marked this conversation as resolved.
Show resolved Hide resolved

| Operator | Meaning |
| -------- | --------|
| ! | not |
| & | and |
| \| | or |

If you are tagging your tests across multiple dimensions, tag expressions help you to select which tests to execute. When tagging by test type (e.g., micro, integration, end-to-end) and feature (e.g., product, catalog, shipping), the following tag expressions can be useful.


| Tag Expression | Selection |
| -------- | --------|
| product | all tests for product |
| catalog \| shipping | all tests for catalog plus all tests for shipping |
| catalog & shipping | all tests that are tagged with both `catalog` and `shipping` tags |
| product & !end-to-end | all tests tagged `product`, except the tests tagged `end-to-end` |
| (micro \| integration) & (product \| shipping) | all micro or integration tests for product or shipping |
lwasylow marked this conversation as resolved.
Show resolved Hide resolved


Taking the last expression above `(micro | integration) & (product | shipping)`

| --%tags |included in run |
| -------- | --------|
| micro | no |
| integration | no |
| micro | no |
| product | no |
| shipping | no |
| micro | no |
| micro, integration | no |
| product, shipping | no |
| micro, product | yes |
| micro, shipping | yes |
| integration, product | yes |
| integration, shipping | yes |
| integration, micro, shipping | yes |
| integration, micro, product | yes |
| integration, shipping ,product | yes |
| micro, shipping ,product | yes |
| integration, micro, shipping ,product | yes |


### Sample execution of test with tags.

Execution of the test with tag expressions is done using the parameter `a_tags`.
Given a test package `ut_sample_test` defined below

```sql linenums="1"
begin
ut.run('hr.test_apply_bonus', a_tags => 'test1,test2');
end;
create or replace package ut_sample_test is

--%suite(Sample Test Suite)
--%tags(api)

--%test(Compare Ref Cursors)
--%tags(complex,fast)
procedure ut_refcursors1;

--%test(Run equality test)
--%tags(simple,fast)
procedure ut_test;

end ut_sample_test;
/

create or replace package body ut_sample_test is

procedure ut_refcursors1 is
v_actual sys_refcursor;
v_expected sys_refcursor;
begin
open v_expected for select 1 as test from dual;
open v_actual for select 2 as test from dual;

ut.expect(v_actual).to_equal(v_expected);
end;

procedure ut_test is
begin
ut.expect(1).to_equal(0);
end;

end ut_sample_test;
/
```

```sql linenums="1"
select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api'));
```
The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api`

```sql linenums="1"
select * from table(ut.run(a_tags => 'fast&complex'));
```
The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex` and `fast`

```sql linenums="1"
select * from table(ut.run(a_tags => 'fast'));
```
The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_test.ut_test` tests, as both tests are tagged with `fast`

### Excluding tests/suites by tags

It is possible to exclude parts of test suites with tags.
In order to do so, prefix the tag name to exclude with a `!` (exclamation) sign when invoking the test run which is equivalent of `-` (dash) in legacy notation.
Examples (based on above sample test suite)

```sql linenums="1"
select * from table(ut.run('hr.test_apply_bonus', a_tags => 'suite1'))
select * from table(ut.run(a_tags => '(api|fast)&!complex'));
lwasylow marked this conversation as resolved.
Show resolved Hide resolved
```

You can also exclude specific tags by adding a `-` (dash) in front of the tag
or

```sql linenums="1"
select * from table(ut.run('hr.test_apply_bonus', a_tags => '-suite1'))
select * from table(ut.run(a_tags => '(api|fast)&!complex&!test1'));
lwasylow marked this conversation as resolved.
Show resolved Hide resolved
```

which is equivalent of exclusion on whole expression

```sql linenums="1"
select * from table(ut.run(a_tags => '(api|fast)&!(complex|test1)'));
```

The above calls will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex` and except those suites/contexts/tests that are marked as `test1`.
Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed.


### Sample execution with `any` and `none`

Given a sample test package:

```sql linenums="1"
create or replace package ut_sample_test is

--%suite(Sample Test Suite)

--%test(Compare Ref Cursors)
--%tags(complex,fast)
procedure ut_refcursors1;

--%test(Run equality test)
--%tags(simple,fast)
procedure ut_test;

--%test(Run equality test no tag)
procedure ut_test_no_tag;

end ut_sample_test;
/

create or replace package body ut_sample_test is

procedure ut_refcursors1 is
v_actual sys_refcursor;
v_expected sys_refcursor;
begin
open v_expected for select 1 as test from dual;
open v_actual for select 2 as test from dual;

ut.expect(v_actual).to_equal(v_expected);
end;

procedure ut_test is
begin
ut.expect(1).to_equal(0);
end;

procedure ut_test_no_tag is
begin
ut.expect(1).to_equal(0);
end;

end ut_sample_test;
/
```

```sql linenums="1"
select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'none'));
```

The above call will execute tests `ut_test_no_tag`

```sql linenums="1"
select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'any'));
```

The above call will execute tests `ut_test` and `ut_refcursors1`

```sql linenums="1"
select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'none|simple'));
```

The above call will execute tests `ut_test_no_tag` and `ut_test`

```sql linenums="1"
select * from table(ut.run(a_tags => 'none|!simple'));
```

The above call will execute tests `ut_test_no_tag` and `ut_refcursors1`

## Keeping uncommitted data after test-run

utPLSQL by default runs tests in autonomous transaction and performs automatic rollback to assure that tests do not impact one-another and do not have impact on the current session in your IDE.
Expand Down
13 changes: 3 additions & 10 deletions 13 source/api/ut_runner.pkb
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,7 @@ create or replace package body ut_runner is
l_run ut_run;
l_coverage_schema_names ut_varchar2_rows;
l_paths ut_varchar2_list;
l_random_test_order_seed positive;
l_tags ut_varchar2_rows := ut_varchar2_rows();
l_random_test_order_seed positive;
begin
ut_event_manager.initialize();
if a_reporters is not empty then
Expand Down Expand Up @@ -118,12 +117,6 @@ create or replace package body ut_runner is
l_coverage_schema_names := ut_suite_manager.get_schema_names(l_paths);
end if;


if a_tags is not null then
l_tags := l_tags multiset union distinct ut_utils.convert_collection(
ut_utils.trim_list_elements(ut_utils.filter_list(ut_utils.string_to_table(a_tags,','),ut_utils.gc_word_no_space))
);
end if;
l_run := ut_run(
a_run_paths => l_paths,
a_coverage_options => ut_coverage_options(
Expand All @@ -140,10 +133,10 @@ create or replace package body ut_runner is
a_test_file_mappings => set(a_test_file_mappings),
a_client_character_set => a_client_character_set,
a_random_test_order_seed => l_random_test_order_seed,
a_run_tags => l_tags
a_run_tags => a_tags
);

ut_suite_manager.configure_execution_by_path(l_paths, l_run.items, l_random_test_order_seed, l_tags);
ut_suite_manager.configure_execution_by_path(l_paths, l_run.items, l_random_test_order_seed, a_tags);
if a_force_manual_rollback then
l_run.set_rollback_type( a_rollback_type => ut_utils.gc_rollback_manual, a_force => true );
end if;
Expand Down
2 changes: 1 addition & 1 deletion 2 source/core/types/ut_run.tpb
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ create or replace type body ut_run as
a_test_file_mappings ut_file_mappings := null,
a_client_character_set varchar2 := null,
a_random_test_order_seed positive := null,
a_run_tags ut_varchar2_rows := null
a_run_tags varchar2 := null
) return self as result is
begin
self.run_paths := a_run_paths;
Expand Down
4 changes: 2 additions & 2 deletions 4 source/core/types/ut_run.tps
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ create or replace type ut_run under ut_suite_item (
project_name varchar2(4000),
items ut_suite_items,
run_paths ut_varchar2_list,
run_tags ut_varchar2_rows,
run_tags varchar2(4000),
coverage_options ut_coverage_options,
test_file_mappings ut_file_mappings,
client_character_set varchar2(100),
Expand All @@ -34,7 +34,7 @@ create or replace type ut_run under ut_suite_item (
a_test_file_mappings ut_file_mappings := null,
a_client_character_set varchar2 := null,
a_random_test_order_seed positive := null,
a_run_tags ut_varchar2_rows := null
a_run_tags varchar2 := null
) return self as result,
overriding member procedure mark_as_skipped(self in out nocopy ut_run,a_skip_reason in varchar2),
overriding member function do_execute(self in out nocopy ut_run) return boolean,
Expand Down
Loading
Morty Proxy This is a proxified and sanitized view of the page, visit original site.