Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion 2 docs/source/components/analyse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Limitations

**Current Limitations:**

- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``) and JSONC (``//``, ``/* */``) comment styles are supported
- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``), JSONC (``//``, ``/* */``) and Markdown (``<!-- … -->``) comment styles are supported
- **Single Comment Style**: Each analysis run processes only one comment style at a time

Extraction Examples
Expand Down
6 changes: 5 additions & 1 deletion 6 docs/source/components/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ Specifies the comment syntax style used in the source code files. This determine

**Type:** ``str``
**Default:** ``"cpp"``
**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``
**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``, ``"markdown"``

.. code-block:: toml

Expand Down Expand Up @@ -326,6 +326,10 @@ Specifies the comment syntax style used in the source code files. This determine
``/* */`` (multi-line)
- ``.jsonc`` (always); ``.json`` only when the file opens with a comment
(e.g. the mode line ``// -*- mode: jsonc -*-``)
* - Markdown
- ``"markdown"``
- ``<!-- … -->`` (HTML-comment block)
- ``.md``, ``.markdown``

.. note:: Future versions may support additional programming languages.

Expand Down
35 changes: 35 additions & 0 deletions 35 docs/source/components/features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,41 @@ Features
.. fault:: Sphinx-codelinks hallucinates traceability objects in JSONC
:id: FAULT_JSONC_2

.. feature:: Markdown Language Support
:id: FE_MARKDOWN

Support for defining traceability objects in Markdown files using
HTML-comment markers (``<!-- @needs … -->``).

The Markdown language parser leverages tree-sitter to identify and extract
standalone HTML-comment blocks from Markdown documents, enabling
requirements traceability in agent definition files, skill files, and other
Markdown-based implementation artefacts.

``.md`` and ``.markdown`` files are auto-discovered when
``comment_type = "markdown"``. Because tree-sitter-markdown exposes
standalone HTML comments as ``html_block`` nodes, only block-level
``<!-- … -->`` markers are captured; inline HTML comments inside paragraphs
are not.

Key capabilities:

* HTML-comment (``<!-- … -->``) detection via tree-sitter
* Auto-discovery of ``.md`` and ``.markdown`` files
* Oneline-only mode (no scope association needed)

.. note::

Because the captured node text includes the full ``<!-- … -->``
delimiters, callers must set ``end_sequence: " -->"`` (not the default
``"\n"``) in their ``oneline_comment_style`` config.

.. fault:: Traceability objects are not detected in Markdown
:id: FAULT_MARKDOWN_1

.. fault:: Sphinx-codelinks hallucinates traceability objects in Markdown
:id: FAULT_MARKDOWN_2

.. feature:: Customized comment styles
:id: FE_CMT

Expand Down
15 changes: 15 additions & 0 deletions 15 docs/source/development/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,21 @@
Changelog
=========

Unreleased
----------

New and Improved
................

- ✨ Added Markdown language support for the ``analyse`` module.

Standalone HTML-comment blocks (``<!-- @needs … -->``) in Markdown files are
now parsed for need ID references and one-line need definitions. ``.md`` and
``.markdown`` files are discovered when ``comment_type = "markdown"``.
Because ``tree-sitter-markdown`` exposes HTML comments as ``html_block``
nodes, callers must set ``end_sequence: " -->"`` in their
``oneline_comment_style`` configuration.

.. _`release:1.3.0`:

1.3.0
Expand Down
1 change: 1 addition & 0 deletions 1 pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ dependencies = [
"tree-sitter-rust>=0.23.0",
"tree-sitter-go>=0.23.0",
"tree-sitter-json>=0.24.8",
"tree-sitter-markdown>=0.5.1",
]

[build-system]
Expand Down
16 changes: 15 additions & 1 deletion 16 src/sphinx_codelinks/analyse/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,11 @@
"type_declaration",
"type_spec",
},
# Markdown has no function/class scopes relevant for code traceability.
# oneline markers in Markdown are always standalone html_block nodes;
# scope association (find_enclosing_scope / find_next_scope) is never
# invoked when get_oneline_needs=True and get_need_id_refs=False.
# CommentType.markdown is intentionally absent from this table.
}

logger = get_logger(__name__)
Expand Down Expand Up @@ -74,6 +79,10 @@
(comment) @comment
"""
JSONC_QUERY = """(comment) @comment"""
# @Markdown HTML-comment query for tree-sitter, IMPL_MD_3, impl, [FE_MARKDOWN]
# Captures block-level HTML nodes (<!-- … -->) as @comment. Inline HTML comments
# inside paragraphs are not captured — only standalone html_block elements.
MARKDOWN_QUERY = """(html_block) @comment"""

# JSON value node types that can be associated with a comment.
JSON_STRUCTURE_TYPES = {
Expand Down Expand Up @@ -103,7 +112,7 @@ def is_text_file(filepath: Path, sample_size: int = 2048) -> bool:
return False


# @Tree-sitter parser initialization for multiple languages, IMPL_LANG_1, impl, [FE_C_SUPPORT, FE_CPP, FE_PY, FE_YAML, FE_RUST, FE_GO, FE_JSONC]
# @Tree-sitter parser initialization for multiple languages, IMPL_LANG_1, impl, [FE_C_SUPPORT, FE_CPP, FE_PY, FE_YAML, FE_RUST, FE_GO, FE_JSONC, FE_MARKDOWN]
def init_tree_sitter(comment_type: CommentType) -> tuple[Parser, Query]:
if comment_type == CommentType.cpp:
import tree_sitter_cpp # noqa: PLC0415
Expand Down Expand Up @@ -140,6 +149,11 @@ def init_tree_sitter(comment_type: CommentType) -> tuple[Parser, Query]:

parsed_language = Language(tree_sitter_json.language())
query = Query(parsed_language, JSONC_QUERY)
elif comment_type == CommentType.markdown:
import tree_sitter_markdown # noqa: PLC0415

parsed_language = Language(tree_sitter_markdown.language())
query = Query(parsed_language, MARKDOWN_QUERY)
else:
raise ValueError(f"Unsupported comment style: {comment_type}")
parser = Parser(parsed_language)
Expand Down
8 changes: 8 additions & 0 deletions 8 src/sphinx_codelinks/source_discover/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@
"rust": ["rs"],
"go": ["go"],
"jsonc": ["jsonc", "json"],
# Markdown uses block-level HTML comments `<!-- @needs … -->` as traceability
# markers. tree-sitter-markdown captures them as `html_block` nodes.
# NOTE: because the node text includes the `<!-- … -->` delimiters, callers
# must set `end_sequence: " -->"` (not the default `"\n"`) in their
# oneline_comment_style config to prevent `-->` from leaking into parsed fields.
"markdown": ["md", "markdown"],
}


Expand All @@ -27,6 +33,8 @@ class CommentType(str, Enum):
go = "go"
# @Support JSONC style comments, IMPL_JSONC_1, impl, [FE_JSONC];
jsonc = "jsonc"
# @Support Markdown HTML-comment style, IMPL_MD_1, impl, [FE_MARKDOWN]
markdown = "markdown"


class SourceDiscoverSectionConfigType(TypedDict, total=False):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"needs": [
{
"id": "IMPL_MD",
"title": "Md Title",
"type": "impl",
"links": {
"links": [
"REQ_MD"
]
},
"metadata": {},
"line": 1
}
],
"need_refs": [],
"marked_rst": [],
"warnings": []
}
7 changes: 7 additions & 0 deletions 7 tests/data/extraction/oneline.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,10 @@ default_oneliner_yaml:
source: |
# @Yaml Title, IMPL_YAML, impl, [REQ_YAML]
key: value

default_oneliner_markdown:
lang: markdown
config:
end_sequence: " -->"
source: |
<!-- @Md Title, IMPL_MD, impl, [REQ_MD] -->
48 changes: 48 additions & 0 deletions 48 tests/test_analyse_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import tree_sitter_cpp
import tree_sitter_go
import tree_sitter_json
import tree_sitter_markdown
import tree_sitter_python
import tree_sitter_rust
import tree_sitter_yaml
Expand Down Expand Up @@ -75,6 +76,53 @@ def init_jsonc_tree_sitter() -> tuple[Parser, Query]:
return parser, query


@pytest.fixture(scope="session")
def init_markdown_tree_sitter() -> tuple[Parser, Query]:
parsed_language = Language(tree_sitter_markdown.language())
query = Query(parsed_language, utils.MARKDOWN_QUERY)
parser = Parser(parsed_language)
return parser, query


@pytest.mark.parametrize(
("code", "expected_count"),
[
# standalone block HTML comment is captured
(
b"<!-- @Md Title, IMPL_MD, impl, [REQ_MD] -->\n",
1,
),
# multiple HTML comment blocks are each captured
(
b"<!-- @Md1, IMPL_MD_1, impl, [REQ_1] -->\n\nSome paragraph.\n\n<!-- @Md2, IMPL_MD_2, impl, [REQ_2] -->\n",
2,
),
# paragraph text without HTML comment produces no comments
(
b"# Heading\n\nJust a paragraph with no markers.\n",
0,
),
],
)
def test_extract_comments_markdown(code, expected_count, init_markdown_tree_sitter):
parser, query = init_markdown_tree_sitter
comments = utils.extract_comments(code, parser, query) or []
assert len(comments) == expected_count


@pytest.mark.parametrize(
"code",
[
b"<!-- @Md Title, IMPL_MD, impl, [REQ_MD] -->\n",
],
)
def test_init_tree_sitter_markdown(code):
"""init_tree_sitter returns a working parser/query pair for markdown."""
parser, query = utils.init_tree_sitter(CommentType.markdown)
comments = utils.extract_comments(code, parser, query)
assert len(comments) == 1


@pytest.mark.parametrize(
("code", "result"),
[
Expand Down
1 change: 1 addition & 0 deletions 1 tests/test_extraction_fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
"yaml": (CommentType.yaml, "yaml"),
"go": (CommentType.go, "go"),
"jsonc": (CommentType.jsonc, "jsonc"),
"markdown": (CommentType.markdown, "md"),
}


Expand Down
3 changes: 2 additions & 1 deletion 3 tests/test_source_discover.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
"comment_type": "java",
},
[
"Schema validation error in field 'comment_type': 'java' is not one of ['cpp', 'cs', 'go', 'jsonc', 'python', 'rust', 'yaml']"
"Schema validation error in field 'comment_type': 'java' is not one of ['cpp', 'cs', 'go', 'jsonc', 'markdown', 'python', 'rust', 'yaml']"
],
),
(
Expand Down Expand Up @@ -182,6 +182,7 @@ def create_source_files(tmp_path: Path) -> Path:
[
("cpp", len(COMMENT_FILETYPE["cpp"])),
("python", len(COMMENT_FILETYPE["python"])),
("markdown", len(COMMENT_FILETYPE["markdown"])),
],
)
def test_comment_filetype(
Expand Down
2 changes: 1 addition & 1 deletion 2 tests/test_src_trace.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
[
"Project 'dcdc' has the following errors:",
"Schema validation error in field 'exclude': 123 is not of type 'string'",
"Schema validation error in field 'comment_type': 'java' is not one of ['cpp', 'cs', 'go', 'jsonc', 'python', 'rust', 'yaml']",
"Schema validation error in field 'comment_type': 'java' is not one of ['cpp', 'cs', 'go', 'jsonc', 'markdown', 'python', 'rust', 'yaml']",
"Schema validation error in field 'gitignore': '_true' is not of type 'boolean'",
"Schema validation error in field 'include': 345 is not of type 'string'",
"Schema validation error in field 'src_dir': ['../dcdc'] is not of type 'string'",
Expand Down
Morty Proxy This is a proxified and sanitized view of the page, visit original site.