[ENH] V1 -> V2 Migration : Runs by Omswastik-11 · Pull Request #1616 · openml/openml-python

Omswastik-11 · Jan 15, 2026

Metadata

Reference Issue:
New Tests Added:
Documentation Updated:
Change Log Entry:

Details

fixes #1624

Signed-off-by: Omswastik-11 <omswastikpanda11@gmail.com>

…into issue1564

codecov-commenter · Jan 15, 2026

Codecov Report

❌ Patch coverage is 38.09524% with 117 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.11%. Comparing base (8ff14eb) to head (fe8f66b).

Files with missing lines	Patch %	Lines
openml/runs/run.py	19.04%	51 Missing ⚠️
openml/_api/config.py	0.00%	30 Missing ⚠️
openml/_api/resources/base/versions.py	8.33%	22 Missing ⚠️
openml/_api/resources/run.py	82.25%	11 Missing ⚠️
openml/runs/functions.py	70.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1616      +/-   ##
==========================================
- Coverage   53.96%   53.11%   -0.86%     
==========================================
  Files          61       62       +1     
  Lines        5051     5202     +151     
==========================================
+ Hits         2726     2763      +37     
- Misses       2325     2439     +114

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Omswastik-11 <omswastikpanda11@gmail.com>

…-11/openml-python into runs-migration-stacked

identified in https://github.com/openml/openml-python/actions/runs/23148612114/job/67243650521?pr=1609

identified in https://github.com/openml/openml-python/actions/runs/23430963986/job/68156944423?pr=1576

geetu040

Please take a look at the CI failures, they look genuine. ~~I think you need to update the tests to use the expected content from local database.~~

I think this is the reason: #1611 (review)

…into runs-migration-stacked

Copilot

Pull request overview

Copilot reviewed 44 out of 47 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · Mar 23, 2026

tests/test_openml/test_config.py

+from urllib.parse import urlparse



urlparse is imported but never used in this test module. This will fail linting in setups that enforce unused imports; please remove it or use it.

Copilot · Mar 23, 2026

tests/test_openml/test_config.py

+def test_get_production_servers():
+    assert openml.config.get_production_servers() == openml.config._get_servers("production")
+
+
+def test_get_test_servers():
+    assert openml.config.get_test_servers() == openml.config._get_servers("test")
+
+
+def test_use_production_servers():
+    openml.config.use_production_servers()
+    servers_1 = openml.config.servers
+
+    openml.config._set_servers("production")
+    servers_2 = openml.config.servers
+
+    assert servers_1 == servers_2
+
+
+def test_use_test_servers():
+    openml.config.use_test_servers()
+    servers_1 = openml.config.servers
+
+    openml.config._set_servers("test")
+    servers_2 = openml.config.servers


These tests pass string literals ("production"/"test") into _get_servers/_set_servers, but those methods are typed/implemented to take ServerMode and will currently raise (or depend on implicit coercion). Use ServerMode.PRODUCTION/ServerMode.TEST here (or update the implementation to accept strings consistently).

Copilot · Mar 23, 2026

openml/_api/resources/run.py

+            V2 server does not yet support POST /runs/ endpoint.
+            Expected availability: Q2 2025


The docstring says the V2 run publish endpoint is expected in “Q2 2025”, which is now in the past. Please update/remove the date to avoid misleading users (e.g., point to an issue/roadmap link instead of a time estimate).

Suggested change

V2 server does not yet support POST /runs/ endpoint.

Expected availability: Q2 2025

V2 server does not yet support the POST /runs/ endpoint.

See the OpenML roadmap or issue tracker for the current status.

Copilot · Mar 23, 2026

openml/_config.py

+def _get_servers(mode: ServerMode) -> dict[APIVersion, dict[str, str | None]]:
+    if mode not in ServerMode:
+        raise ValueError(f'invalid mode="{mode}" allowed modes: {", ".join(list(ServerMode))}')
+    return deepcopy(_SERVERS_REGISTRY[mode])


_get_servers only accepts ServerMode enum values, but it’s being used with strings (e.g., "production"/"test") in the tests and will raise ValueError because mode not in ServerMode for a plain string. Consider accepting str | ServerMode here (e.g., coerce via ServerMode(mode) when isinstance(mode, str)), or update all call sites to pass ServerMode.PRODUCTION/ServerMode.TEST consistently.

Suggested change

def _get_servers(mode: ServerMode) -> dict[APIVersion, dict[str, str | None]]:

if mode not in ServerMode:

raise ValueError(f'invalid mode="{mode}" allowed modes: {", ".join(list(ServerMode))}')

return deepcopy(_SERVERS_REGISTRY[mode])

def _get_servers(mode: ServerMode | str) -> dict[APIVersion, dict[str, str | None]]:

if isinstance(mode, str):

try:

mode = ServerMode(mode)

except ValueError as exc:

allowed = ", ".join(m.value for m in ServerMode)

raise ValueError(f'invalid mode="{mode}" allowed modes: {allowed}') from exc

elif mode not in ServerMode:

allowed = ", ".join(m.value for m in ServerMode)

raise ValueError(f'invalid mode="{mode}" allowed modes: {allowed}')

return deepcopy(_SERVERS_REGISTRY[cast(ServerMode, mode)])

Copilot · Mar 23, 2026

openml/_config.py

+        return self.servers[self.api_version]["apikey"]
+
+    @apikey.setter
+    def apikey(self, value: str | None) -> None:


The apikey setter no longer enforces the previous type restriction (string or None). Since this is part of the public configuration surface, it’s easy for callers to accidentally set non-strings (e.g., dict/int) and only fail later during request building. Consider restoring a simple type check in the setter to fail fast with a clear TypeError.

Suggested change

def apikey(self, value: str | None) -> None:

def apikey(self, value: str | None) -> None:

if value is not None and not isinstance(value, str):

raise TypeError(

f"apikey must be a string or None, got {type(value).__name__}: {value!r}"

)

Copilot · Mar 23, 2026

tests/test_openml/test_config.py

+    openml.config.server = "temp-server1"
+    openml.config.apikey = "temp-apikey1"
+    openml.config._get_servers(mode)["server"] = 'temp-server2'
+    openml.config._get_servers(mode)["apikey"] = 'temp-server2'


This test is mutating the dict returned by _get_servers(mode) using keys "server"/"apikey", but _get_servers returns a mapping keyed by APIVersion, so these assignments will raise KeyError. If the intent is to verify the returned value is a deep copy, mutate a nested entry (e.g., servers_copy[api_version]["server"] = ...) and then assert a fresh _get_servers(mode) call still equals orig_servers.

Copilot

Pull request overview

Copilot reviewed 10 out of 12 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · Mar 23, 2026

openml/_api/resources/run.py

+        ------
+        OpenMLNotSupportedError
+            V2 server does not yet support POST /runs/ endpoint.
+            Expected availability: Q2 2025


The docstring claims an "Expected availability: Q2 2025" for the V2 publish endpoint, which is now in the past and likely to become increasingly misleading. Consider removing the date or replacing it with a timeless note (e.g., link to an issue/roadmap entry) so the message stays accurate over time.

Suggested change

Expected availability: Q2 2025

Copilot · Mar 23, 2026

tests/test_runs/test_run_functions.py

 import time
 import unittest
 import warnings
+from urllib.parse import urljoin


urljoin is imported but not used anywhere in this test module. Please remove the import (or use it) to avoid lint/test failures in environments enforcing unused-import checks.

Suggested change

from urllib.parse import urljoin

Copilot · Mar 23, 2026

tests/test_runs/test_run_functions.py

+@mock.patch.object(requests.Session, "request")
 def test_delete_run_not_owned(mock_delete, test_files_directory, test_server_v1, test_apikey_v1):
    content_file = test_files_directory / "mock_responses" / "runs" / "run_delete_not_owned.xml"
    mock_delete.return_value = create_request_response(


The patched method is requests.Session.request, but the mock parameter is still named mock_delete, which is misleading. Rename the argument to something like mock_request (and update local variable names) to reflect what’s being mocked.

Copilot · Mar 23, 2026

openml/runs/functions.py

+    ignore_cache : bool
+        Whether to ignore the cache. If ``true`` this will download and overwrite the run xml
+        even if the requested run is already cached.



The get_run docstring repeats the ignore_cache parameter section twice. Please remove the duplicate block so the docstring stays accurate and easier to maintain.

Suggested change

ignore_cache : bool

Whether to ignore the cache. If ``true`` this will download and overwrite the run xml

even if the requested run is already cached.

Copilot · Mar 23, 2026

openml/_api/resources/run.py

+                f'"http://openml.org/openml": {runs_dict}',
+            )
+
+        assert isinstance(runs_dict["oml:runs"]["oml:run"], list), type(runs_dict["oml:runs"])


assert isinstance(..., list) is used for validating the parsed XML shape. Asserts can be disabled with Python optimizations (-O), which would skip this check and potentially lead to harder-to-debug errors later. Prefer raising a TypeError/ValueError (similar to openml.runs.functions.__list_runs) instead of using assert for input validation.

Suggested change

assert isinstance(runs_dict["oml:runs"]["oml:run"], list), type(runs_dict["oml:runs"])

if not isinstance(runs_dict["oml:runs"]["oml:run"], list):

raise TypeError(

"Error in return XML, 'oml:runs/oml:run' is expected to be a list, "

f"but got {type(runs_dict['oml:runs']['oml:run']).__name__}: "

f"{runs_dict['oml:runs']['oml:run']}"

)

Copilot

Pull request overview

Copilot reviewed 10 out of 12 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · Mar 23, 2026

openml/_api/resources/run.py

+        assert isinstance(runs_dict["oml:runs"]["oml:run"], list), type(runs_dict["oml:runs"])
+


_parse_list_xml uses a bare assert to validate that oml:run is a list. Assertions can be stripped with python -O, and even when enabled they raise AssertionError (less actionable for API consumers). Prefer an explicit isinstance(..., list) check that raises TypeError/ValueError with a clear message (similar to openml.runs.functions.__list_runs).

Copilot · Mar 23, 2026

tests/test_api/test_run.py

+    try:
+        run_v1.get(run_id=TEST_RUN_ID)
+        return TEST_RUN_ID
+    except Exception:
+        runs_df = run_v1.list(limit=1, offset=0)
+        if runs_df.empty:
+            pytest.skip("No runs available on configured test server")
+        return int(runs_df.iloc[0]["run_id"])


_get_any_run_id catches a broad Exception, which can hide real test failures (e.g., network issues, parsing errors) and make debugging harder. It would be more robust to only fall back to list() for the expected failure modes (e.g., an OpenML server “not found”/HTTP error for that specific run_id) and let unexpected exceptions fail the test.

geetu040

Please see #1619 (review) regarding changed cache path

for more information, see https://pre-commit.ci

geetu040

are you sure main was cleanly merged into this? I doubt that.
you haven't moved the tests/files correctly, please see #1619 (comment)
since you opened the other PR as well, if any feedback is relevant from here for there or vice-versa, please apply

geetu040 · Mar 25, 2026

openml/_api/resources/base/versions.py

merge was not clean here, right?

geetu040 · Mar 25, 2026

openml/_api/resources/base/versions.py


+    def _parse_xml_response(self, payload: bytes | str, **kwargs: Any) -> Mapping[str, Any]:
+        try:
+            return cast("Mapping[str, Any]", xmltodict.parse(payload, **kwargs))


is this casting really needed? I don't think so

geetu040 · Mar 25, 2026

openml/_api/resources/base/versions.py

+                raise
+
+            xml_text = payload_text[xml_start:]
+            return cast("Mapping[str, Any]", xmltodict.parse(xml_text, **kwargs))


same as above

geetu040 · Mar 25, 2026

openml/_api/config.py

remove this file

geetu040 · Mar 25, 2026

openml/runs/functions.py

+            run_id,
+            reset_cache=ignore_cache,
+        ),
+    )


no need for explicit cast here

geetu040 · Mar 25, 2026

tests/test_api/test_run.py

+        runs_df = run_v1.list(limit=1, offset=0)
+        if runs_df.empty:
+            pytest.skip("No runs available on configured test server")
+        return int(runs_df.iloc[0]["run_id"])


I think this function can be avoided, it's not clean. what's the problem with a fixed run id?

geetu040 · Mar 25, 2026

tests/test_api/test_run.py

+@pytest.mark.test_server()
+def test_run_v1_v2_contracts(run_v1, run_v2):
+    run_id = _get_any_run_id(run_v1)
+
+    run_from_v1 = run_v1.get(run_id=run_id)
+    _assert_run_shape(run_from_v1)
+
+    with pytest.raises(OpenMLNotSupportedError, match="does not support `get`"):
+        run_v2.get(run_id=run_id)
+
+    with pytest.raises(OpenMLNotSupportedError, match="does not support `list`"):
+        run_v2.list(limit=5, offset=0)
+
+    with pytest.raises(OpenMLNotSupportedError, match="does not support `publish`"):
+        run_v2.publish(path="run", files={"description": "<run/>"})


geetu040 · Mar 25, 2026

tests/test_runs/test_run_functions.py

geetu040 · Mar 25, 2026

tests/test_runs/test_run_functions.py

        self._test_local_evaluations(run)

    @pytest.mark.sklearn()
-    @pytest.mark.skip("https://github.com/openml/openml-python/issues/1586")


but it's unrelated to this PR right? why remove the skip?

geetu040 · Mar 25, 2026

tests/test_runs/test_run_functions.py


        self._test_local_evaluations(run)

-    @pytest.mark.skip(reason="https://github.com/openml/openml-python/issues/1586")


same as above

geetu040

I've update #1619 (review). Please sync with main and update cache files accordingly.

geetu040 · Mar 25, 2026

openml/_api/resources/run.py

+        int
+            The ID of the published run.
+        """
+        return super().publish(path=path, files=files)


no need to implement publish here in RunV1API

geetu040 and others added 2 commits January 15, 2026 14:51

Merge branch 'main' into migration

5762185

revert the tarsks/function changes

3da4760

Signed-off-by: Omswastik-11 <omswastikpanda11@gmail.com>

geetu040 mentioned this pull request Jan 15, 2026

[ENH] V1 → V2 API Migration #1575

Open

18 tasks

satvshr and others added 9 commits January 15, 2026 21:36

made requested changes

021a1e1

Merge branch 'main' into issue1564

4c4a12c

made requested changes

1d91220

Merge branch 'issue1564' of https://github.com/satvshr/openml-python …

3e26ace

…into issue1564

fixed bugs

0060b2e

fixed bugs

65ba66b

fixed bugs

317c6e9

fixed bugs

503ab82

Merge branch 'main' into runs-migration-stacked

e5b7af4

satvshr added 2 commits January 16, 2026 15:36

Merge branch 'main' into issue1564

fd7ea2b

bug fixing

fa3cd40

geetu040 assigned Omswastik-11 Jan 19, 2026

Omswastik-11 and others added 14 commits January 19, 2026 19:43

migrating v1 -> v2 runs

716667c

Signed-off-by: Omswastik-11 <omswastikpanda11@gmail.com>

Merge branch 'runs-migration-stacked' of https://github.com/Omswastik…

68a403a

…-11/openml-python into runs-migration-stacked

Merge branch 'main' into migration

7e9bc1f

add tests directory

c603383

use enum for delay method

ff6a8b0

implement cache

f01898f

refactor clients

5c4511e

Merge branch 'main' into issue1564

9d0098f

fix import in resources/base.py

43276d2

refactor and add exception handling

1206f69

Merge branch 'main' into issue1564

bde5942

refactor resources/base/

4948e99

implement delete

a354167

implement publish and minor refactoring

1fe7e3e

geetu040 added 6 commits March 16, 2026 20:12

create enum ServerMode

7d61107

update config for ServerMode

1ecbbba

update tests for ServerMode

65472ed

udpate apikey in _TEST_SERVERS_LOCAL

44b48b5

fix: remove duplicate server name in cache path

04bc83b

identified in https://github.com/openml/openml-python/actions/runs/23148612114/job/67243650521?pr=1609

test: remove check for ":" since windows CI expects it

f926092

identified in https://github.com/openml/openml-python/actions/runs/23430963986/job/68156944423?pr=1576

geetu040 suggested changes Mar 23, 2026

View reviewed changes

Omswastik-11 added 2 commits March 23, 2026 16:54

Merge branch 'migration' of https://github.com/geetu040/openml-python …

aa91d92

…into runs-migration-stacked

updated the tests

22d7179

Copilot AI review requested due to automatic review settings March 23, 2026 17:39

Copilot started reviewing on behalf of Omswastik-11 March 23, 2026 17:40 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Omswastik-11 added 2 commits March 23, 2026 23:16

Merge branch 'main' into runs-migration-stacked

dc081ca

updated the test

808bc5b

Copilot AI review requested due to automatic review settings March 23, 2026 18:02

Copilot started reviewing on behalf of Omswastik-11 March 23, 2026 18:02 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Omswastik-11 added 2 commits March 24, 2026 00:02

updated the tests

61816f4

updated the tests

c02e09b

Copilot AI review requested due to automatic review settings March 23, 2026 18:57

Copilot started reviewing on behalf of Omswastik-11 March 23, 2026 18:58 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

geetu040 reviewed Mar 24, 2026

View reviewed changes

Omswastik-11 and others added 2 commits March 24, 2026 23:02

Merge branch 'main' into runs-migration-stacked

19c7dab

[pre-commit.ci] auto fixes from pre-commit.com hooks

fe8f66b

for more information, see https://pre-commit.ci

Copilot AI review requested due to automatic review settings March 24, 2026 17:32

Omswastik-11 review requested due to automatic review settings March 24, 2026 17:32

geetu040 suggested changes Mar 25, 2026

View reviewed changes

		V2 server does not yet support POST /runs/ endpoint.
		Expected availability: Q2 2025

-def _get_servers(mode: ServerMode) -> dict[APIVersion, dict[str, str | None]]:
-    if mode not in ServerMode:
-        raise ValueError(f'invalid mode="{mode}" allowed modes: {", ".join(list(ServerMode))}')
-    return deepcopy(_SERVERS_REGISTRY[mode])
+def _get_servers(mode: ServerMode | str) -> dict[APIVersion, dict[str, str | None]]:
+    if isinstance(mode, str):
+        try:
+            mode = ServerMode(mode)
+        except ValueError as exc:
+            allowed = ", ".join(m.value for m in ServerMode)
+            raise ValueError(f'invalid mode="{mode}" allowed modes: {allowed}') from exc
+    elif mode not in ServerMode:
+        allowed = ", ".join(m.value for m in ServerMode)
+        raise ValueError(f'invalid mode="{mode}" allowed modes: {allowed}')
+    return deepcopy(_SERVERS_REGISTRY[cast(ServerMode, mode)])

-    def apikey(self, value: str | None) -> None:
+    def apikey(self, value: str | None) -> None:
+        if value is not None and not isinstance(value, str):
+            raise TypeError(
+                f"apikey must be a string or None, got {type(value).__name__}: {value!r}"
+            )

	ignore_cache : bool
	Whether to ignore the cache. If ``true`` this will download and overwrite the run xml
	even if the requested run is already cached.

-        assert isinstance(runs_dict["oml:runs"]["oml:run"], list), type(runs_dict["oml:runs"])
+        if not isinstance(runs_dict["oml:runs"]["oml:run"], list):
+            raise TypeError(
+                "Error in return XML, 'oml:runs/oml:run' is expected to be a list, "
+                f"but got {type(runs_dict['oml:runs']['oml:run']).__name__}: "
+                f"{runs_dict['oml:runs']['oml:run']}"
+            )

		assert isinstance(runs_dict["oml:runs"]["oml:run"], list), type(runs_dict["oml:runs"])


		self._test_local_evaluations(run)

		@pytest.mark.skip(reason="https://github.com/openml/openml-python/issues/1586")

Search code, repositories, users, issues, pull requests...

Uh oh!

Conversation

Omswastik-11 commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Metadata

Details

Uh oh!

codecov-commenter commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

geetu040 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

geetu040 left a comment

Choose a reason for hiding this comment

Uh oh!

geetu040 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Omswastik-11 commented Jan 15, 2026 •

edited

Loading

codecov-commenter commented Jan 15, 2026 •

edited

Loading

geetu040 left a comment •

edited

Loading