Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Add Python extension using pocketpy#124

Open
hcarty wants to merge 8 commits into
orx:masterorx/orx:masterfrom
hcarty:pocketpy-python-extensionhcarty/orx:pocketpy-python-extensionCopy head branch name to clipboard
Open

Add Python extension using pocketpy#124
hcarty wants to merge 8 commits into
orx:masterorx/orx:masterfrom
hcarty:pocketpy-python-extensionhcarty/orx:pocketpy-python-extensionCopy head branch name to clipboard

Conversation

@hcarty
Copy link
Copy Markdown
Member

@hcarty hcarty commented May 24, 2025

This adds Python bindings to orx through an extension, using https://pocketpy.dev/ to provide a fully embedded Python runtime.

To quote the pocketpy website:

pocketpy is a portable Python 3.x interpreter, written in C11. It aims to be an alternative to Lua for game scripting, with elegant syntax, powerful features and competitive performance. pocketpy has no dependencies other than the C standard library, which can be easily integrated into your C/C++ project. Developers are able to write Python bindings via C-API

Extension features

  • Adds a +python option to orx's project init tool
    • This will initialize a new project so that it can run fully through Python, including init, update, and exit engine callbacks and several event handlers.
  • Event handlers
    • Object creation, deletion
    • Physics collision, separation
    • Setting shader parameter values
  • Bindings to several engine functions under an orx.* module namespace, including:
    • orxObject_* in orx.object
    • orxConfig_* in orx.config
    • orxInput_* in orx.input
    • orxVector_* in orx.vector
    • orxClock_ComputeDT in orx.clock
    • orxCommand_Evaluate(WithGUID) in orx.command
    • orxLOG as orx.log
  • *.pyi files for each of the provided Python modules to improve support for editor integration like VSCode's Python extension and support Python type checkers like pyright and mypy
  • Registers Python.Exec command to run Python code from commands
  • A Scroll-inspired pyscroll module to bind per-object behaviors
  • Uses orxResource module for loading *.py sources. Python modules can be included from any supported resource location, including bundles when the bundle extension is used.
  • Works for native and web/emscripten builds
  • The full pocketpy runtime and standard library is compiled directly into the application binary with no additional runtime requirements
  • Custom function and data bindings can be added using pocketpy's C API

@hcarty hcarty mentioned this pull request May 24, 2025
@hcarty hcarty force-pushed the pocketpy-python-extension branch from 588a49c to 979f215 Compare May 24, 2025 02:03
Copy link
Copy Markdown
Member Author

@hcarty hcarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding some review comments.

@@ -0,0 +1,2 @@
[tool.pyright]
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used by pyright, a type checker for Python. There are no *.py sources for the core engine bindings so the type checker will warn or error on some settings otherwise.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't this be initialized programmatically instead?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can remove the configuration file. It's something that was more necessary when I started the extension, but the tooling has advanced since then and people may want to use other tooling for their own projects.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By that I meant, could this be expressed in an orx config file instead, and set programmatically when starting the extension?
This way we keep all configuration stuff consistent across engine & extensions.

Comment thread code/build/template/build/premake4.lua
Comment thread code/build/template/build/premake4.lua
Comment thread code/build/template/build/premake4.lua Outdated
Comment thread code/build/template/data/[+python python]/[=name].py
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pocketpy's header. The only modification from upstream is the reference to orx's memory allocation functions.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pocketpy's implementation. It is taken directly from upstream without modification.

Copy link
Copy Markdown
Contributor

@iarwain iarwain May 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be moved inside include/extensions, and included directly from orxPy.h.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started with that but it breaks compilation of C++ projects since the code does not compile as C++.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, in that case let's move it inside the newly added extensions subdirectory.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice change - thank you! I moved src/pocketpy/ into src/extensions/pocketpy/.

Comment thread code/build/template/src/[=name].c[+c++ pp]
]
[+python
// Initialize Python support
orxPy_InitVM();
Copy link
Copy Markdown
Member Author

@hcarty hcarty May 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initialize the pocketpy virtual machine. This does not execute any game-specific Python code. It does define all of the orx.* engine module bindings.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This calls the user's/game's orx.init function, called from the init project's main source file.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this call also be inside orxExtensions.h, in InitExtensions()?
Is there a reason why it is separate from orxPy_InitVM()?

Copy link
Copy Markdown
Member Author

@hcarty hcarty May 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started with this combined as well, but if a project defines its own modules from C then those need to be defined before any game Python code is run. Splitting the two initializes the VM early, then gives space for custom C bindings and module definitions.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the VM rely on any orx API? If not, we could move its initialization to the BootStrap function instead.
I'd rather not have any extension code outside of orxExtensions.h as much as possible as it breaks adding/removing extensions (the main source file can't be touched once created).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Short version of all the text below: I think we can make this change. I have some minor concerns around more advanced use cases described below. If it's ok for me to change the InitExtensions return type then I think my main concerns would be addressed.

The call to orxPy_Init here is like the call to orxPy_Update and orxPy_CameraUpdate earlier in the file. They each call Python functions defined by game code at the engine-appropriate time. I've tested locally and orxPy_Init could be called from InitExtensions if orxPy_InitVM is called from BootstrapExtensions.

I have some concerns with that change:

  • InitExtensions doesn't return a status value. Would it be ok to have InitExtensions return orxSTATUS rather than void? In the PR's current state a failure during orxInit_Py will stop the game from starting up. I think that's a good feature to have but I'm not sure if others would agree.
  • It obfuscates where the game's Python code is loaded and the user's orx.init callback function is called. I think that's ok. The Python extension is kind of like Scroll in that it can be used to change where primary game logic is run (C vs C++ vs C++/Scroll vs Python). However the Python extension uses the existing C/C++/Scroll engine callbacks for bootstrap/init/update/camera update rather than defining its own.
  • It is less clear where a user's custom C to Python bindings should be added. If orxPy_Init is called from InitExtensions then those modules must be defined before the call to InitExtensions. That should work as long as those module definitions do not rely on other extensions being initialized first. Custom module definitions shouldn't require that - initialization should happen when the game logic is initialized. I think that's ok as well.

A bit of extra context on the extension's two init functions:

orxPy_InitVM initializes the pocketpy VM and registers the orx.* Python modules with that VM. It does not run any Python code or call any orx APIs. It could be called from BootstrapExtensions.

orxPy_Init loads and runs the user's [=name].py code, sets up any references or bindings between user code and the pocketpy VM, and calls the user-defined Python init function if one has been defined. Arbitrary orx and Python code can be run at this point.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll have to check orxPy.h, but shouldn't it no rely on Scroll at all and be orthogonal to it, so as to have a single code flow no matter what the kind of project is?

@hcarty hcarty force-pushed the pocketpy-python-extension branch 8 times, most recently from 97aba70 to c5a7d73 Compare May 27, 2025 23:09
@hcarty hcarty force-pushed the pocketpy-python-extension branch 2 times, most recently from 6b8e77b to 7710397 Compare June 7, 2025 05:11
@hcarty hcarty force-pushed the pocketpy-python-extension branch 2 times, most recently from 95acc75 to c251a20 Compare June 14, 2025 21:40
@hcarty hcarty force-pushed the pocketpy-python-extension branch 3 times, most recently from ba19856 to 369a643 Compare August 30, 2025 04:27
@hcarty hcarty marked this pull request as draft September 6, 2025 12:55
@hcarty hcarty force-pushed the pocketpy-python-extension branch from 369a643 to 0aa62c0 Compare September 7, 2025 21:57
@hcarty hcarty force-pushed the pocketpy-python-extension branch from 0aa62c0 to ac40c04 Compare September 30, 2025 15:56
@hcarty hcarty force-pushed the pocketpy-python-extension branch 3 times, most recently from dc1b963 to 8db24ea Compare October 19, 2025 03:19
@hcarty hcarty force-pushed the pocketpy-python-extension branch from 8db24ea to 638c2bb Compare November 3, 2025 05:37
@hcarty hcarty force-pushed the pocketpy-python-extension branch 3 times, most recently from feda00f to 20af680 Compare November 29, 2025 06:31
@hcarty hcarty force-pushed the pocketpy-python-extension branch from 20af680 to d84797c Compare December 7, 2025 15:51
@hcarty hcarty force-pushed the pocketpy-python-extension branch from d84797c to 529e69d Compare December 15, 2025 01:54
@hcarty hcarty force-pushed the pocketpy-python-extension branch 4 times, most recently from 9471b2d to 2032939 Compare February 9, 2026 05:59
@hcarty hcarty force-pushed the pocketpy-python-extension branch 2 times, most recently from af16e1c to a2a66a0 Compare February 21, 2026 23:01
@hcarty hcarty force-pushed the pocketpy-python-extension branch 2 times, most recently from 41e2226 to c71a53d Compare March 3, 2026 04:35
@hcarty hcarty marked this pull request as ready for review March 6, 2026 04:51
Copy link
Copy Markdown
Member Author

@hcarty hcarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few more small review comments.

Comment thread code/build/template/build/premake4.lua Outdated
configuration {"*Profile*"}
targetsuffix ("p")
defines {"__orxPROFILER__"}
defines {"__orxPROFILER__"[+python ,"NDEBUG"]}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NDEBUG signals to pocketpy that this is not a debug build.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can add a separate defines line for Python, it might make it slightly more easy to read.
Or just decompose the existing one on multiple lines.

Comment thread code/build/template/build/premake4.lua
orx.on_camera_update = camera_update

# Setup engine callbacks for miniscroll
miniscroll.setup(init, update)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

miniscroll has its own registration logic to simplify what needs to be done by the user.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we give it a more obvious name maybe, like PyScroll?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like PyScroll! I started with "mini" as the prefix because of its relatively limited scope, but I think PyScroll is better and disambiguates well with Scroll.

@hcarty hcarty force-pushed the pocketpy-python-extension branch from c71a53d to 689ae8a Compare March 11, 2026 03:27
@hcarty hcarty force-pushed the pocketpy-python-extension branch from 689ae8a to 01fe4ae Compare March 13, 2026 01:50
@iarwain
Copy link
Copy Markdown
Contributor

iarwain commented Mar 21, 2026

  • A Scroll-inspired miniscroll module to bind per-object behaviors

Wouldn't pyscroll be a better suited name in that case?

  • The full pocketpy runtime and standard library is compiled directly into the application binary with no additional runtime requirements

Love that part! :)

Additionally, would there be any benefit to expose orx's new task worker pool to the Python environment?

Copy link
Copy Markdown
Contributor

@iarwain iarwain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments. :)

Comment thread code/build/template/build/premake4.lua
Comment thread code/build/template/build/premake4.lua

// clang-format off

#define PK_VERSION "2.1.0"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor alignment issue. Is this file auto-generated?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file comes from the pocketpy project. pocketpy generates the combined pocketpy.h and pocketpy.c files from individual header and source files as part of the release process.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll have to check orxPy.h, but shouldn't it no rely on Scroll at all and be orthogonal to it, so as to have a single code flow no matter what the kind of project is?

@@ -0,0 +1,2 @@
[tool.pyright]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By that I meant, could this be expressed in an orx config file instead, and set programmatically when starting the extension?
This way we keep all configuration stuff consistent across engine & extensions.

[+inspector
OnCreate = Inspector.RegisterObject ^]
[+python
Input = @
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use the TriggerList here as well?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

miniscroll/pyscroll does not support the same . and other prefixes that Scroll does, so it can't match the behavior the scroll provides with the TriggerList. It's something I could probably add, but last time I looked there were some features Scroll relies on for this that were not bound by orxpy.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, so the inputs are handled entirely in Python code then?

Fine by me, but I do find the data-only input handling handy for simple situations and I would recommend implementing something similar to what's inside ScrollBase (it's not much code).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the inputs are handled entirely in Python code then?

Somewhat - it uses a similar approach to what is done in Scroll, but not as refined. This is run for each orx object registered with pyscroll/miniscroll on each update:

      # Input triggers
      for input_name in mini.input_names:
        if orx.input.has_been_activated(input_name):
          mini.o.fire_trigger("Input", refinement=[input_name])

The mini.input_names list is populated when the object is created, based on the input set associated with the object, similar to what's done by Scroll but captured on object creation rather than looking up the inputs at runtime.

I agree the additional functionality and flexibility provided by Scroll is worth having here. I'll work on that.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can add a separate defines line for Python, it might make it slightly more easy to read.
Or just decompose the existing one on multiple lines.

I decomposed the existing one on multiple lines and am happy to change the approach further if you prefer.

]
[+python
// Call Python update
[-scroll orxPy_Update(_pstClockInfo, orxNULL);][+scroll orxPy_Update(&_rstClockInfo, orxNULL);]]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the only difference is the first parameter, you can have the -scroll/+scroll only for that parameter instead, for brevity.

]
[+python
// Call Python camera update
[-scroll orxPy_CameraUpdate(_pstClockInfo, orxNULL);][+scroll orxPy_CameraUpdate(&_rstClockInfo, orxNULL);]]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar suggestion as before: use -scroll/+scroll only around the first parameter.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done for both.

Comment thread code/build/template/src/[=name].c[+c++ pp]

]
[+scroll
[-python
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the -python blocks could be merged in a single one and the callbacks registered at the very end instead.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

- PR style feedback
@hcarty
Copy link
Copy Markdown
Member Author

hcarty commented Mar 22, 2026

Wouldn't pyscroll be a better suited name in that case?

I think that's reasonable. I went back and forth in my own head a lot about the naming for this. pyscroll is probably a better name.

I currently use Mini as the name for the ScrollObject equivalent in Python. Would you be ok with me using ScrollObject as the class name in pyscroll as well? Or would you prefer a separate name like PyScroll or PyScrollObject to avoid ambiguity in discussions about code?

  • The full pocketpy runtime and standard library is compiled directly into the application binary with no additional runtime requirements

Love that part! :)

I do too! That's the main incentive I had to try this out in the first place. No separate runtime to worry about, or binary compatibility issues.

Additionally, would there be any benefit to expose orx's new task worker pool to the Python environment?

There may be, but I don't think it's needed at this point.

pocketpy supports multiple VMs running in parallel, with one VM per thread. It uses C11 threading, manages its own threads, and that does work with this extension. None of the orx modules are available from any VM other than the main VM though, to avoid issues with the config system and object creation.

The primitives pocketpy provides for communication and tasking between VMs are very minimal so I eventually want to add some friendly wrappers on top. Those may be modeled after the task worker pool.

@hcarty
Copy link
Copy Markdown
Member Author

hcarty commented Mar 22, 2026

Wouldn't pyscroll be a better suited name in that case?

I think that's reasonable. I went back and forth in my own head a lot about the naming for this. pyscroll is probably a better name.

I currently use Mini as the name for the ScrollObject equivalent in Python. Would you be ok with me using ScrollObject as the class name in pyscroll as well? Or would you prefer a separate name like PyScroll or PyScrollObject to avoid ambiguity in discussions about code?

Thinking about this more, I don't think ScrollObject is a good choice on the Python side since the Scroll and pocketpy extensions can be freely mixed in projects - and my hope is that both will be used side by side in at least some cases.

Maybe something like Base or Bound? I haven't found a name I'm completely happy with yet.

- Renamed miniscroll to pyscroll
- Renamed Mini class to Base
@hcarty
Copy link
Copy Markdown
Member Author

hcarty commented Mar 22, 2026

I renamed Mini to Base and miniscroll to pyscroll in 3fd6ff6.

- Add orx.input.has_new_status (orxInput_HasNewStatus)
- orx.object.fire_trigger returns event success/failure status
- pyscroll now more closely matches Scroll's config-based input trigger handling
@hcarty
Copy link
Copy Markdown
Member Author

hcarty commented Mar 24, 2026

d2afc53 brings pyscroll's config-based input handling more inline with what Scroll supports, including . and - prefixes and input value refinements.

- Fix pyscroll input trigger event handling
- Add orx.input.get_next binding orxInput_GetNext
Copy link
Copy Markdown
Contributor

@iarwain iarwain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes. Let's merge it post 1.17 release!

@iarwain iarwain self-assigned this Apr 7, 2026
- De-conflict Scroll and pyscroll input handling by not registering the Logo object with pyscroll in a fresh init project with both +python and +scroll
- Refactor pyscroll to be a bit cleaner and more friendly to common Python idioms
- Added and expanded doc comments for pyscroll
- Update to latest pocketpy
@hcarty
Copy link
Copy Markdown
Member Author

hcarty commented Apr 10, 2026

Thank you for the review!

I pushed that refactor that I mentioned on discord in d72da6a. I'm happy to have this merged after the 1.17 release!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's super minor, but given this already follows most of orx's coding conventions, would you mind going the extra mile and prefixing function parameters with _?
For example:
orxCHAR *orxPy_ReadSourceFromMain(const orxSTRING _zPath, int *_piDataSize)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly - apologies for the inconsistency for those function parameter names. I'll fix that and push an updated commit in a few minutes.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style fixes are in 687f589. There were a lot! Thank you for calling the naming mismatch out. I'd like this to feel like orx as much as possible.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking care of it!

hcarty added 2 commits April 10, 2026 14:23
- Fix function parameter names to match orx's coding conventions
- Bugfix in pyscroll's shader parameter handling
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.