Closed
Description
I found that the Python 3.11.1 implementation of all() is 30% slower compared to Python 3.10.9.
any() also seems to be around 4% slower on my device
Environment
- CPython versions tested on: Python 3.10.9 and Python 3.11.1
- Operating system and architecture: Windows 10 22H2 (Build 19045) 64 bit
Code to test all():
import timeit
from functools import partial
import platform
def find_by_keys(
keys: list[str],
table: list[dict[str, str | int]],
match_data: dict[str, str | int],
) -> dict[str, str | int] | None:
for item in table:
if all(item[k] == match_data[k] for k in keys):
return item
return None
def main():
keys: list[str] = ["id", "key_1", "key_2"]
table: list[dict[str, str | int]] = [
{
"id": i,
"key_1": "val_1",
"key_2": "val_2",
}
for i in range(1, 5001)
]
match_data: dict[str, str | int] = {
"id": 3000,
"key_1": "val_1",
"key_2": "val_2",
}
timeit_output = timeit.repeat(
partial(find_by_keys, keys, table, match_data), repeat=10000, number=1
)
average_time = sum(timeit_output) / len(timeit_output)
best_time = min(timeit_output)
tps_output = 1 / average_time
print(f"Python version = {platform.python_version()}")
print(f"Average time = {average_time}")
print(f"Best time = {best_time}")
print(f"Average transactions per second = {tps_output}")
if __name__ == "__main__":
main()Results for all()
Console output using Python 3.10.9:
Python version = 3.10.9
Average time = 0.0008256170657486655
Best time = 0.0007106999401003122
Average transactions per second = 1211.2152733824669
Console output using Python 3.11.1:
Python version = 3.11.1
Average time = 0.0011819988898839802
Best time = 0.001033599954098463
Average transactions per second = 846.0244832363215
Code to test any():
import timeit
from functools import partial
import platform
def find_by_keys(
keys: list[str],
table: list[dict[str, str | int]],
match_data: dict[str, str | int],
) -> dict[str, str | int] | None:
for item in table:
if any(item[k] == match_data[k] for k in keys):
return item
return None
def main():
keys: list[str] = ["id", "key_1", "key_2"]
table: list[dict[str, str | int]] = [
{
"id": i,
"key_1": "foo_1",
"key_2": "foo_2",
}
for i in range(1, 5001)
]
match_data: dict[str, str | int] = {
"id": 3000,
"key_1": "val_1",
"key_2": "val_2",
}
timeit_output = timeit.repeat(
partial(find_by_keys, keys, table, match_data), repeat=10000, number=1
)
average_time = sum(timeit_output) / len(timeit_output)
best_time = min(timeit_output)
tps_output = 1 / average_time
print(f"Python version = {platform.python_version()}")
print(f"Average time = {average_time}")
print(f"Best time = {best_time}")
print(f"Average transactions per second = {tps_output}")
if __name__ == "__main__":
main()Results for any()
Console output using Python 3.10.9:
Python version = 3.10.9
Average time = 0.0009300541202770546
Best time = 0.00090230000205338
Average transactions per second = 1075.206246817238
Console output using Python 3.11.1:
Python version = 3.11.1
Average time = 0.0009645268900785595
Best time = 0.000930099980905652
Average transactions per second = 1036.7777303943815
Linked PRs
- gh-100762: Switch order of exception match in gen_close #101011
- GH-100762: Don't call
gen.throw()ingen.close(), unless necessary. #101013 - [3.11] GH-100762: Don't call gen.throw() in gen.close(), unless necessary #101316
- gh-100762: Fix optimization in gen_close #111069
- [3.12] gh-100762: Fix optimization in gen_close (GH-111069) #115818
Metadata
Metadata
Assignees
Labels
only security fixesonly security fixesonly security fixesonly security fixes(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)Performance or resource usagePerformance or resource usage