Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 40f8ee6

Browse filesBrowse files
rvermeulenrvermeulen
committed
Add minimal test case for deviation report generation
1 parent 5571453 commit 40f8ee6
Copy full SHA for 40f8ee6

File tree

Expand file treeCollapse file tree

6 files changed

+138
-0
lines changed
Filter options
Expand file treeCollapse file tree

6 files changed

+138
-0
lines changed

‎scripts/reports/analysis_report_test.py

Copy file name to clipboardExpand all lines: scripts/reports/analysis_report_test.py
+25Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
from pathlib import Path
33
import sys
44
from guideline_recategorizations import generate_guideline_recategorizations_report
5+
from deviations import generate_deviations_report
56

67
script_path = Path(__file__)
78
# Add the shared modules to the path so we can import them.
@@ -34,4 +35,28 @@ def test_guideline_recategorizations_report(tmp_path):
3435
expected = expected.replace("$codeql-version$", codeql.version).replace("$database-path$", str(db_path))
3536
actual = (tmp_path / "guideline_recategorizations_report.md").read_text()
3637

38+
assert(expected == actual)
39+
40+
def test_deviations_report(tmp_path):
41+
42+
db_path = tmp_path / 'test-db'
43+
src_root = TEST_DATA_DIR / 'deviations'
44+
codeql = CodeQL()
45+
46+
compile_src_command = "clang -fsyntax-only test.cpp"
47+
index_coding_standards_config_command = f"python3 {SCRIPTS_DIR}/configuration/process_coding_standards_config.py"
48+
49+
try:
50+
codeql.create_database(src_root, 'cpp', db_path, compile_src_command, index_coding_standards_config_command)
51+
except CodeQLError as err:
52+
print(err.stdout)
53+
print(err.stderr)
54+
raise err
55+
56+
generate_deviations_report(db_path, REPO_ROOT, tmp_path)
57+
58+
expected = (TEST_DATA_DIR / 'deviations' / 'deviations_report.md.expected').read_text()
59+
expected = expected.replace("$codeql-version$", codeql.version).replace("$database-path$", str(db_path))
60+
actual = (tmp_path / "deviations_report.md").read_text()
61+
3762
assert(expected == actual)

‎scripts/reports/test-data/deviations/coding-standards.yml

Copy file name to clipboardExpand all lines: scripts/reports/test-data/deviations/coding-standards.yml
Whitespace-only changes.

‎scripts/reports/test-data/deviations/deviations_report.md.expected

Copy file name to clipboard
+48Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Deviations report
2+
3+
## Overview
4+
5+
- Report generated with supported CodeQL version $codeql-version$
6+
- Database path: $database-path$
7+
- 3 valid deviation records and 14 invalid deviation records found in the database
8+
- 3 valid deviation permits and 2 invalid deviation permits found in the database
9+
10+
## Deviation Records
11+
12+
| Rule ID | Query ID | Automated Scope | Scope | Justification | Background | Requirements
13+
| --- | --- | --- | --- | --- | --- | --- |
14+
| A0-1-1 | cpp/autosar/useless-assignment | Applies to the following file paths: invalid | | | | |
15+
| A0-1-1 | cpp/autosar/useless-assignment | Applies to the following file paths: valid | | This useless assignment is required. | | |
16+
| A0-4-2 | cpp/autosar/type-long-double-used | Identified by the use of the code-identifier: a-0-4-2-deviation | | long double is required for interaction with third-party libraries. | | |
17+
18+
## Invalid Deviation Records
19+
| Path | Reason |
20+
| --- | --- |
21+
| invalid/coding-standards.xml:5:7:5:26 | No rule-id and query-id specified for this deviation record. |
22+
| invalid/coding-standards.xml:6:7:8:26 | The rule-id `bad rule id` for this deviation matches none of the available queries. |
23+
| invalid/coding-standards.xml:9:7:11:26 | A query-id of `bad rule id` is specified for this deviation, but not rule-id is specified. |
24+
| invalid/coding-standards.xml:15:7:17:26 | A query-id of `cpp/autosar/useless-assignment` is specified for this deviation, but not rule-id is specified. |
25+
| invalid/coding-standards.xml:22:7:26:26 | A deviation `raised-by` is specified without providing an `approved-by`. |
26+
| invalid/coding-standards.xml:22:7:26:26 | A deviation `raised-by` is specified without providing both a `name` and `date`. |
27+
| invalid/coding-standards.xml:27:7:33:26 | A deviation `raised-by` is specified without providing an `approved-by`. |
28+
| invalid/coding-standards.xml:27:7:33:26 | A deviation `raised-by` is specified without providing both a `name` and `date`. |
29+
| invalid/coding-standards.xml:34:7:41:26 | A deviation `raised-by` is specified without providing an `approved-by`. |
30+
| invalid/coding-standards.xml:42:7:50:26 | A deviation `approved-by` is specified without providing both a `name` and `date`. |
31+
| invalid/coding-standards.xml:51:7:61:26 | A deviation `approved-by` is specified without providing both a `name` and `date`. |
32+
| invalid/coding-standards.xml:74:7:78:26 | There is no deviation permit with id `non-existing-permit`. |
33+
| invalid/coding-standards.xml:79:7:81:26 | No rule-id and query-id specified for this deviation record. |
34+
| invalid/coding-standards.xml:85:7:88:26 | The deviation is applied to a query with the rule category 'mandatory' that does not permit a deviation. |
35+
36+
## Deviation Permits
37+
38+
| Permit ID | Rule ID | Query ID | Automated Scope | Scope | Justification | Background | Requirements
39+
| --- | --- | --- | --- | --- | --- | --- | --- |
40+
| DP1 | | | Application depends on the associated deviation records | | foo bar baz | | |
41+
| DP2 | A0-1-1 | cpp/autosar/useless-assignment | Application depends on the associated deviation records | | | | |
42+
| DP3 | | | Application depends on the associated deviation records | | | | |
43+
44+
## Invalid Deviation Permits
45+
| Path | Reason |
46+
| --- | --- |
47+
| invalid/coding-standards.xml:100:7:103:33 | Deviation permit does not specify a permit identifier. |
48+
| invalid/coding-standards.xml:104:7:107:33 | Deviation permit specifies unknown property `invalid-property`. |

‎scripts/reports/test-data/deviations/invalid/coding-standards.yml

Copy file name to clipboard
+58Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
deviations:
2+
-
3+
- rule-id: bad rule id
4+
- query-id: bad rule id
5+
- rule-id: A0-1-1
6+
- query-id: cpp/autosar/useless-assignment
7+
- rule-id: A0-1-1
8+
query-id: cpp/autosar/useless-assignment
9+
- rule-id: A0-1-1
10+
query-id: cpp/autosar/useless-assignment
11+
raised-by:
12+
- rule-id: A0-1-1
13+
query-id: cpp/autosar/useless-assignment
14+
raised-by:
15+
name: foo1
16+
- rule-id: A0-1-1
17+
query-id: cpp/autosar/useless-assignment
18+
raised-by:
19+
name: foo2
20+
date: 1970-01-01Z
21+
- rule-id: A0-1-1
22+
query-id: cpp/autosar/useless-assignment
23+
raised-by:
24+
name: foo3
25+
date: 1970-01-01Z
26+
approved-by:
27+
- rule-id: A0-1-1
28+
query-id: cpp/autosar/useless-assignment
29+
raised-by:
30+
name: foo4
31+
date: 1970-01-01Z
32+
approved-by:
33+
name: bar1
34+
- rule-id: A0-1-1
35+
query-id: cpp/autosar/useless-assignment
36+
raised-by:
37+
name: foo5
38+
date: 1970-01-01Z
39+
approved-by:
40+
name: bar2
41+
date: 1970-01-01Z
42+
- rule-id: A0-1-1
43+
query-id: cpp/autosar/useless-assignment
44+
permit-id: non-existing-permit
45+
- permit-id: DP1
46+
- permit-id: DP2
47+
- rule-id: RULE-13-6
48+
query-id: c/misra/sizeof-operand-with-side-effect
49+
deviation-permits:
50+
- permit-id: DP1
51+
justification: foo bar baz
52+
- permit-id: DP2
53+
rule-id: A0-1-1
54+
query-id: cpp/autosar/useless-assignment
55+
- rule-id: A0-1-1
56+
query-id: cpp/autosar/useless-assignment
57+
- permit-id: DP3
58+
invalid-property: invalid-property

‎scripts/reports/test-data/deviations/test.cpp

Copy file name to clipboardExpand all lines: scripts/reports/test-data/deviations/test.cpp
Whitespace-only changes.

‎scripts/reports/test-data/deviations/valid/coding-standards.yml

Copy file name to clipboard
+7Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
deviations:
2+
- rule-id: A0-1-1
3+
query-id: cpp/autosar/useless-assignment
4+
justification: This useless assignment is required.
5+
- rule-id: A0-4-2
6+
justification: long double is required for interaction with third-party libraries.
7+
code-identifier: a-0-4-2-deviation

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.