Extension:Scribunto

MediaWiki extensions manual

Scribunto Release status: stable
Implementation	Parser extension
Description	Provides a framework for embedding scripting languages into MediaWiki pages
Author(s)	Victor and Tim (Tim Starlingtalk)
Latest version	Continuous updates
MediaWiki	1.20 and up (check out the REL1_20 branch for support with 1.20, but note it lacks some features)
PHP	5.3+
Database changes	No
License	GPL-2.0+ AND MIT
Download	Download snapshot (Git master) Git [?]: repo summary browse file tree commit history code review
Namespace	Module
Parameters $wgScribuntoDefaultEngine $wgScribuntoEngineConf $wgScribuntoUseGeSHi $wgScribuntoUseCodeEditor
Hooks used SoftwareInfo ParserFirstCallInit ParserLimitReport ParserClearState ParserCloned CanonicalNamespaces CodeEditorGetPageLanguage EditPageBeforeEditChecks EditPageBeforeEditButtons EditFilterMerged ArticleViewHeader ContentHandlerDefaultModelFor UnitTestsList
Translate the Scribunto extension if it is available at translatewiki.net
Check usage and version matrix; code metrics
Issues:	Open tasks · Report a bug

The Scribunto (Latin: "they shall write") extension allows for embedding scripting languages in MediaWiki. Currently the only supported scripting language is Lua.

License

This extension contains code licensed GNU General Public License v2.0 or later (GPL-2.0+) as well as code licensed MIT License (MIT).

Requirements

PCRE version compatibility

PCRE 8.33 or later is recommended. You can see the version of PCRE used by PHP by viewing a phpinfo() web page, or from the command line with the command php -i | grep 'PCRE'.

• Scribunto will not work with versions of PCRE lower than 8.10.
• PCRE 8.31 has bugs with JIT pattern studying that breaks various patterns when used with HHVM.
• PCRE 8.32 has a bug that will cause it to reject certain non-character codepoints, which will cause errors in the mw.html module.

CentOS 6 and RHEL 6 are stuck on PCRE 7 and need to be upgraded.

Updating to 8.33 on a server with an older version may be relatively complicated. See Updating to PCRE 8.33 or Higher for details.

PHP mbstring extension

PHP needs to have the mbstring extension enabled.

To see if it installed on your server, you can check if it is enabled with the server command line: php -m

Lua binary

Bundled binaries

Scribunto comes bundled with Lua binary distributions for Linux (x86 and x86-64), Mac OS X Lion, and Windows (32- and 64-bit).

Scribunto should work for you out of the box if:

1. Your web server is run on one of the above platforms.
2. PHP's proc_open function is not restricted
3. Your web server is configured to allow the execution of binary files in the MediaWiki tree.

 Note: Execute permissions may need to be set; for example, in Linux use

chmod a+x /path/to/extensions/Scribunto/engines/LuaStandalone/binaries/lua5_1_5_linux_64_generic/lua

Additional binaries

Additional Lua binary distributions, which may be needed for your web server if its operating system is not in the list above, can be obtained from http://luabinaries.sourceforge.net/ or from your Linux distribution. Only binary files for Lua 5.1.x are supported. Once you've installed the appropriate binary file on your web server, configure the location of the file with:

# where lua is the name of the binary file
# e.g. sourceforge LuaBinaries 5.1.5 - Release 2 name the binary file lua5.1
$wgScribuntoEngineConf['luastandalone']['luaPath'] = '/path/to/binaries/lua5.1';

Note that you should not add the above line unless you've confirmed that Scribunto's built-in binaries don't work for you.

Installation

• If using VagrantMediaWiki-Vagrant, install with vagrant enable-role scribunto && vagrant provision

Manual installation

• Download and place the file(s) in a directory called Scribunto in your extensions/ folder.
• Add the following code at the bottom of your LocalSettings.php:

require_once "$IP/extensions/Scribunto/Scribunto.php";
$wgScribuntoDefaultEngine = 'luastandalone';

• Set execute permissions for the Lua binaries bundled with this extension:

chmod a+x /path/to/extensions/Scribunto/engines/LuaStandalone/binaries/yourOS/lua

• Set type to httpd_sys_script_exec_t if SELinux is enforced:

chcon -t httpd_sys_script_exec_t /path/to/extensions/Scribunto/engines/LuaStandalone/binaries/yourOS/lua

• Y Done - Navigate to Special:Version on your wiki to verify that the extension is successfully installed.

Optional Installation

Integrating extensions

For a more pleasant user interface, with syntax highlighting and a code editor with autoindent, install the following extensions:

• Extension:WikiEditor
• Extension:SyntaxHighlight
• Extension:CodeEditor

Then in your LocalSettings.php after all the extension registrations, add:

$wgScribuntoUseGeSHi = true;
$wgScribuntoUseCodeEditor = true;

LuaSandbox

We have developed a PHP extension written in C called LuaSandbox. It can be used as an alternative to the standalone binary, and will provide improved performance.

Download

git clone https://gerrit.wikimedia.org/r/p/mediawiki/php/luasandbox.git

• Or download a snapshot and unpack

Requirements

Install the headers and library files for PHP and either Lua 5.1.x or LuaJIT 1.1.x.

• For Debian-derived Linux distributions, such as Ubuntu:

sudo apt-get install php5-dev
sudo apt-get install liblua5.1-dev

• For Centos/Redhat-derived Linux distributions:

sudo yum install php-devel lua lua-devel

• For Mac OS X:

brew install lua

Installation

For php-fpm or PHP module of Apache

cd luasandbox
phpize && ./configure && make && sudo make install

Then make sure that there is file luasandbox.ini at /etc/php5/conf.d with extension=luasandbox.so inside. After Apache or php-fpm restart, the extension should be reported by phpinfo () and Special:Version.

For HHVM

cd luasandbox
hphpize && cmake . && sudo make install

Under Debian systems, you could need to replace cmake . with cmake . -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None -DLUA_USE_CPP=1.

Then add

hhvm.enable_zend_compat = true
hhvm.dynamic_extension_path=/usr/lib/x86_64-linux-gnu/hhvm/extensions/20150212
hhvm.dynamic_extensions[luasandbox]=luasandbox.so

to /etc/hhvm/php.ini. You may need to replace the path to luasandbox.so (/usr/lib/x86_64-linux-gnu/hhvm/extensions/20150212) with whatever install path was reported by make install.

After executing sudo service hhvm restart, the extension should be present in get_loaded_extensions () and reported by Special:Version.

php.ini

• Add the following extension reference: extension=luasandbox.so
• Restart the web server
• Verify that there is a luasandbox section on the phpinfo() page
  • if you use nginx, make sure you edit the fpm php.ini file (e.g. /etc/php5/fpm/php.ini in Ubuntu)
• Some PHP installations have two php.ini files, see Manual:php.ini

Configuration

The following configuration variables are available:

$wgScribuntoDefaultEngine 

Select the engine. Valid values are the keys in $wgScribuntoEngineConf, which by default are 'luasandbox' or 'luastandalone'.

$wgScribuntoUseGeSHi 

When Extension:SyntaxHighlight is installed, set this true to use it when displaying Module pages.

$wgScribuntoUseCodeEditor 

When Extension:CodeEditor is installed, set this true to use it when editing Module pages.

$wgScribuntoEngineConf 

An associative array for engine configuration. Keys are the valid values for $wgScribuntoDefaultEngine, and values are associative arrays of configuration data. Each configuration array must contain a 'class' key naming the ScribuntoEngineBase subclass to use.

Logging

Error output produced by the standalone interpreter are not logged by default. Configure logging with:

$wgScribuntoEngineConf['luastandalone']['errorFile'] = '/path/to/file.log';

LuaStandalone

The following keys are used in $wgScribuntoEngineConf for Scribunto_LuaStandaloneEngine. Generally you'd set these as something like

$wgScribuntoEngineConf['luastandalone']['key'] = 'value';

luaPath 

Specify the path to a Lua interpreter.

errorFile 

Specify the path to a file, writable by the web server user, where the error and debugging output from the standalone interpreter will be logged.

memoryLimit 

Specify the memory limit in bytes for the standalone interpreter on Linux (enforced using ulimit).

cpuLimit 

Specify the CPU time limit in seconds for the standalone interpreter on Linux (enforced using ulimit).

allowEnvFuncs 

Set true to allow use of setfenv and getfenv in modules.

LuaSandbox

The following keys are used in $wgScribuntoEngineConf for Scribunto_LuaSandboxEngine. Generally you'd set these as something like

$wgScribuntoEngineConf['luasandbox']['key'] = 'value';

memoryLimit 

Specify the memory limit in bytes.

cpuLimit 

Specify the CPU time limit in seconds.

profilerPeriod 

Specify the time between polls in sections for the Lua profiler.

allowEnvFuncs 

Set true to allow use of setfenv and getfenv in modules.

Usage

Scripts go in a new namespace called Module. Each module has a collection of functions, which can be called using wikitext syntax such as:

 {{#invoke: Module_name | function_name | arg1 | arg2 | arg3 ... }}

Lua

Learning Lua

Lua is a simple programming language intended to be accessible to beginners. For a quick crash-course on Lua, try Learn Lua in 15 Minutes.

The best comprehensive introduction to Lua is the book Programming in Lua. The first edition (for Lua 5.0) is available online and is mostly relevant to Lua 5.1, the version used by Scribunto:

• Programming in Lua (scroll down past the book ads to find the text)

The reference manual is also useful:

• Lua reference manual

Lua environment

In Lua, the set of all global variables and functions is called an environment.

Each {{#invoke:}} call runs in a separate environment. Variables defined in one {{#invoke:}} will not be available from another. This restriction was necessary to maintain flexibility in the wikitext parser implementation.

The environment which scripts run in is not quite the same as in standard Lua. These differences are noted in Special:MyLanguage/Extension:Scribunto/Lua reference manual#Differences from standard Lua.

Troubleshooting

Troubleshooting using the clickable "Script error" link.

Note that red messages are clickable and will provide more detailed information.

Lua error: Internal error: The interpreter exited with status 1

When using the LuaStandalone engine (this is the default), errors along the lines of "Lua error: Internal error: The interpreter exited with status 1" may be generated if the standalone Lua interpreter cannot be executed or runs into various runtime errors. To obtain more information, assign a file path to $wgScribuntoEngineConf['luastandalone']['errorFile']. The interpreter's error output will be logged to the specified file, which should prove more helpful in tracking down the issue. The information in the debug log includes debugging information, which is why there is so much of it. You should be able to ignore any line beginning with "TX" or "RX".

Lua error: Internal error: The interpreter exited with status 2

When using the LuaStandalone engine (this is the default), status 2 suggests memory allocation errors, probably caused by settings that allocate inadequate memory space for PHP or lua, or both. Assigning a file path to $wgScribuntoEngineConf['luastandalone']['errorFile'] and examining that output can be valuable in diagnosing memory allocation errors.

Increase PHP allocation in your PHP configuration; add the line memory_limit = 200M. This allocation of 200MB is often sufficient (as of MediaWiki 1.24) but can be increased as required. Set Scribunto's memory allocation in LocalSettings.php as a line:

  $wgScribuntoEngineConf['luastandalone']['memoryLimit'] = 209715200; # bytes

Finally, depending on the server configuration, some installations may be helped by adding another LocalSettings.php line

  $wgMaxShellMemory = 204800; # in KB

Note that all 3 memory limits are given in different units.

Lua error: Internal error: The interpreter exited with status 126

When using the LuaStandalone engine (this is the default), errors along the lines of "Lua error: Internal error: The interpreter exited with status 126" may be generated if the standalone Lua interpreter cannot be executed. This generally arises from either of 2 causes:

• The lua executable file's permissions do not include Execute. Set permissions as described under #Installation.
• The server does not allow execution of files from the place where the executable is installed, e.g. the filesystem is mounted with the 'noexec' flag. This often occurs with shared hosted servers. Remedies include adjusting $wgScribuntoEngineConf['luastandalone']['luaPath'] to point to a Lua 5.1 binary installed in an executable location, or adjusting or convincing the shared host to adjust the setting preventing execution.

Error condition such as: Fatal exception of type MWException

Check the MediaWiki, PHP, or webserver logs for more details on the exception, or temporarily set $wgShowExceptionDetails to true.

version `GLIBC_2.11' not found

If the above gives you errors such as "version `GLIBC_2.11' not found", it means the version of the standard C library on your system is too old for the binaries provided with Scribunto. You should upgrade your C library, or use a version of Lua 5.1 compiled for the C library you do have installed. To upgrade your C library, your best option is usually to follow your distribution's instructions for upgrading packages (or for upgrading to a new release of the distribution, if applicable).

If you copy the lua binaries from Scribunto master (or from gerrit:77905), that should suffice, if you can't or don't want to upgrade your C library. The distributed binaries were recently recompiled against an older version of glibc, so the minimum is now 2.3 rather than 2.11.

Lua errors in Scribunto files

Errors here include:

• attempt to index field 'text' (a nil value)
• Lua error in mw.html.lua at line 253: Invalid class given:

If you are getting errors such these when attempting to use modules imported from WMF wikis, most likely your version of Scribunto is out of date. Upgrade if possible; for advanced users, you might also try to identify the needed newer commits and cherry-pick them into your local installation.

preg_replace_callback(): Compilation failed: unknown property name after \P or \p at offset 7

preg_replace_callback(): Compilation failed: unknown property name after \P or \p at offset 7

• this usually indicates an incompatible version of PCRE; you’ll need to update to >= 8.10
• @todo: link to instructions on how to upgrade

Lua error

If you copy templates from Wikipedia and then get big red "Lua error: x" messages where the Scribunto invocation (e.g. the template that uses {{#invoke:}}) should be, that probably means that you didn't import everything you needed. Make sure that you tick the "Include templates" box at w:Special:Export when you export.

When importing pages from another wiki, it is also possible for templates or modules in the imported data to overwrite existing templates or modules with the same title, which may break existing pages, templates, and modules that depend on the overwritten versions.

Blank screen

Make sure your extension version is applicable to your MediaWiki version.

Design documents

• Parser interface design
• Victor's API proposal
• Documentation specification
• Tim's draft roadmap

Other pages

• Deployment priorities
• Brainstorming
• Lua scripting - Wikimedia activity page describing deployment plan to Wikimedia sites
• Lua reference manual - The reference about the Lua language, as well as its standard libraries and common Scribunto modules supported on Wikimedia sites
• Lua 5.2 changes - A list of known changes in Lua 5.2 that may cause code written in 5.1 to function unexpectedly
• Example modules
• Example extension - Code for example extensions extending the Scribunto library.
• We use Lua

See also

• Extension:Capiunto - Provides basic Infobox functionality for the Scribunto extension.
• Extension:Wikibase Client - Provides Wikibase (part of Wikidata project) Lua client functionality for the Scribunto extension.
• wikipedia:Help:Lua
• wikidata:Help:Lua

Notes