Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Ctypes implementation #2364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 57 commits into
base: main
Choose a base branch
Loading
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
41b30de
metaclass support for #[pyclass] macro
youknowone Aug 7, 2021
b4a1b80
Initial ctypes module structure with dlopen and dlsym
rodrigocam Oct 29, 2020
b9ada0a
Refactoring ctypes module to a PyPy-like file tree
darleybarreto Oct 31, 2020
1fa7e80
Add __getattr__ to deal with attributes at runtime
rodrigocam Oct 31, 2020
b30dbae
Adding a very basic and initial implementation of _SimpleCData and Fu…
darleybarreto Nov 2, 2020
89587f0
Add PyFuncPtr tp_new from DLL and basic tp_call
rodrigocam Nov 3, 2020
b4ddf4d
Adding basic functions to call libffi-rs
darleybarreto Nov 4, 2020
9177473
Fix compile errors
rodrigocam Nov 5, 2020
14b08b9
Changing some pieces of SharedLibrary
darleybarreto Nov 7, 2020
8adf4dd
Fixing some ref problems in functions.rs
darleybarreto Nov 8, 2020
4341c97
Cahnge PyRc to PyRef in data cache
rodrigocam Nov 10, 2020
eb8541c
Fixing arg type casting
darleybarreto Nov 10, 2020
e38b0a8
Refactoring PyCFuncPtr
darleybarreto Nov 12, 2020
c2df5de
Moving dlsym to from_dll
darleybarreto Nov 12, 2020
fa6b19b
Adding proper *mut c_void casting
darleybarreto Nov 13, 2020
7d220c3
Adding 'reopen' lib
darleybarreto Nov 13, 2020
0a92095
Adding function call
darleybarreto Nov 13, 2020
8ceffd9
Fixing clippy warnings
darleybarreto Nov 14, 2020
4c585ab
Fixing dangling ref
darleybarreto Nov 15, 2020
b09ebe0
Starting primitive types impl
darleybarreto Nov 19, 2020
d7888f9
Adding metaclass
darleybarreto Nov 19, 2020
145afc7
Adding PyCDataMethods trait
darleybarreto Nov 21, 2020
aff291f
Adding default value for PySimpleType
darleybarreto Nov 22, 2020
d29c3bf
Adding some comments
darleybarreto Nov 23, 2020
6fdffab
Implement PySimpleType __init__
darleybarreto Nov 26, 2020
568c838
Modifying Buffer for PyCData
darleybarreto Nov 29, 2020
fb07e53
Adding methods for PyCDataMethods
darleybarreto Nov 30, 2020
6122e40
Adding PyCData_NewGetBuffer related code
darleybarreto Dec 4, 2020
6a4f881
Fixing small fixes
darleybarreto Dec 4, 2020
e2e0ac8
Testing PySimpleType basic functionalities
darleybarreto Dec 5, 2020
c1324ce
Refactoring SharedLibrary
darleybarreto Dec 5, 2020
3f40466
Fixing several bugs
darleybarreto Dec 5, 2020
3e52625
Fixing function call
darleybarreto Dec 6, 2020
3454751
Fixing more of function calls
darleybarreto Dec 6, 2020
0f25f4d
PySimpleType from_param initial commit
darleybarreto Dec 7, 2020
c4f27fb
Adding more methods to PySimpleType
darleybarreto Dec 7, 2020
b88e544
Minor fixes to get compiling on master
coolreader18 Dec 9, 2020
4ce1123
Use static_cell for libcache
coolreader18 Dec 13, 2020
e528fe7
Adding RawBuffer & reworking low level function call
darleybarreto Dec 19, 2020
420a67a
Small fixes
darleybarreto Dec 21, 2020
d656650
Initial commit for PyCArray
darleybarreto Dec 21, 2020
2a1f6d7
Reworking CData buffers
darleybarreto Dec 30, 2020
ed44269
Adding PyCArray setitem
darleybarreto Dec 31, 2020
4d2c678
Adding some helper functions and initial tests
darleybarreto Dec 31, 2020
2c440b3
Adding PyCArray's from_param
darleybarreto Jan 1, 2021
0b65e0d
Adding several changes to make some tests pass
darleybarreto Jan 2, 2021
7c529af
Fix build on master
youknowone Aug 6, 2021
fe01b1a
hide ctypes test
youknowone Aug 6, 2021
25e0403
clean up ctypes::array
youknowone Aug 7, 2021
efabe9b
skeleton PyCSimpleType
youknowone Aug 7, 2021
b79477a
submodule extension for ctypes::dll
youknowone Aug 7, 2021
2904015
Merge pull request #4 from youknowone/dll-submodule
darleybarreto Aug 7, 2021
66a91cb
Add all tests from CPython and PyPy
darleybarreto Aug 8, 2021
577c5ae
Add suggestions and bump dependencies
darleybarreto Aug 8, 2021
6eb0794
Starting to add Metas to primitive and array ctypes
darleybarreto Aug 8, 2021
5916218
Fixing some terribly wrong impls and bugs.
darleybarreto Aug 15, 2021
c2ba116
Fix compilation
darleybarreto Aug 21, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixing clippy warnings
  • Loading branch information
darleybarreto authored and youknowone committed Aug 7, 2021
commit 8ceffd9f327a5eefda6c929ab7f3758f7e4fd0d9
161 changes: 107 additions & 54 deletions 161 vm/src/stdlib/ctypes/common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,21 @@ extern crate lazy_static;
extern crate libffi;
extern crate libloading;

use ::std::{collections::HashMap, mem, os::raw::*};
use ::std::{collections::HashMap, mem, os::raw::*, ptr};

use libffi::low::{
call as ffi_call, ffi_abi_FFI_DEFAULT_ABI as ABI, ffi_cif, ffi_type, prep_cif, CodePtr,
Error as FFIError,
};
use libffi::middle;
use libloading::Library;
use num_bigint::BigInt;

use crate::builtins::PyTypeRef;
use crate::common::lock::PyRwLock;
use crate::pyobject::{PyObjectRc, PyObjectRef, PyRef, PyResult, PyValue, StaticType};
use crate::pyobject::{
PyObjectRc, PyObjectRef, PyRef, PyResult, PyValue, StaticType, TryFromObject,
};
use crate::VirtualMachine;

pub const SIMPLE_TYPE_CHARS: &str = "cbBhHiIlLdfuzZqQP?g";
Expand Down Expand Up @@ -82,6 +85,60 @@ fn str_to_type(ty: &str) -> *mut ffi_type {
)
}

fn py_to_ffi(ty: *mut *mut ffi_type, obj: PyObjectRef, vm: &VirtualMachine) -> *mut c_void {
match_ffi_type!(
unsafe { *ty },
c_schar => {
let mut r = i8::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void

}
c_int => {
let mut r = i32::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
c_short => {
let mut r = i16::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
c_ushort => {
let mut r = u16::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
c_uint => {
let mut r = u32::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
//@ TODO: Convert c*longlong from BigInt?
c_long | c_longlong => {
let mut r = i64::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
c_ulong | c_ulonglong => {
let mut r = u64::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
f32 => {
let mut r = f32::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
f64 | longdouble=> {
let mut r = f64::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
c_uchar => {
let mut r = u8::try_from_object(vm, obj).unwrap();
&mut r as *mut _ as *mut c_void
}
pointer => {
usize::try_from_object(vm, obj).unwrap() as *mut c_void
}
void => {
ptr::null_mut()
}
)
}

#[derive(Debug)]
pub struct Function {
pointer: *const c_void,
Expand Down Expand Up @@ -116,7 +173,7 @@ impl Function {
arg_ptrs: Vec<PyObjectRef>,
vm: &VirtualMachine,
) -> PyResult<PyObjectRc> {
let mut return_type: *mut ffi_type = &mut unsafe { self.return_type.read() };
let return_type: *mut ffi_type = &mut unsafe { self.return_type.read() };

let result = unsafe {
prep_cif(
Expand All @@ -136,14 +193,18 @@ impl Function {
return Err(vm.new_runtime_error("The ABI is invalid or unsupported".to_string()));
}

let mut argument_pointers: Vec<*mut c_void> = arg_ptrs
.iter()
.zip(self.arguments.iter_mut())
.map(|(o, t)| {
let tt: *mut *mut ffi_type = t;
py_to_ffi(tt, o.clone(), vm)
})
.collect();

let cif_ptr = &self.cif as *const _ as *mut _;
let fun_ptr = CodePtr::from_ptr(self.pointer);
let mut args_ptr = self
.arguments
.iter_mut()
.map(|p: &mut *mut ffi_type| p as *mut _ as *mut c_void)
.collect()
.as_mut_ptr();
let args_ptr = argument_pointers.as_mut_ptr();

let ret_ptr = unsafe {
match_ffi_type!(
Expand All @@ -170,31 +231,25 @@ impl Function {
}
c_long => {
let r: c_long = ffi_call(cif_ptr, fun_ptr, args_ptr);
vm.new_pyobj(r as u64)
vm.new_pyobj(r as i64)
}
c_longlong => {
let r: c_longlong = ffi_call(cif_ptr, fun_ptr, args_ptr);
vm.new_pyobj(r as i64)
// vm.new_pyobj(r as i128)
vm.new_pyobj(BigInt::from(r as i128))
}
c_ulong => {
let r: c_ulong = ffi_call(cif_ptr, fun_ptr, args_ptr);
vm.new_pyobj(r as u64)
}
c_ulonglong => {
let r: c_ulonglong = ffi_call(cif_ptr, fun_ptr, args_ptr);
vm.new_pyobj(r as u64)
// vm.new_pyobj(r as u128)
vm.new_pyobj(BigInt::from(r as u128))
}
f32 => {
let r: c_float = ffi_call(cif_ptr, fun_ptr, args_ptr);
vm.new_pyobj(r as f32)
}
f64 => {
let r: c_double = ffi_call(cif_ptr, fun_ptr, args_ptr);
vm.new_pyobj(r as f64)
}
longdouble => {
f64 | longdouble => {
let r: c_double = ffi_call(cif_ptr, fun_ptr, args_ptr);
vm.new_pyobj(r as f64)
}
Expand All @@ -207,7 +262,6 @@ impl Function {
vm.new_pyobj(r as *const _ as usize)
}
void => {
let r: c_void = ffi_call(cif_ptr, fun_ptr, args_ptr);
vm.ctx.none()
}
)
Expand All @@ -224,8 +278,7 @@ unsafe impl Sync for Function {}
#[derive(Debug)]
pub struct SharedLibrary {
path_name: String,
lib: Library,
is_open_g: Box<bool>,
lib: Option<Library>,
}

impl PyValue for SharedLibrary {
Expand All @@ -238,35 +291,36 @@ impl SharedLibrary {
pub fn new(name: &str) -> Result<SharedLibrary, libloading::Error> {
Ok(SharedLibrary {
path_name: name.to_string(),
lib: Library::new(name.to_string())?,
is_open_g: Box::new(true),
lib: Some(Library::new(name.to_string())?),
})
}

pub fn get_sym(&self, name: &str) -> Result<*mut c_void, libloading::Error> {
pub fn get_sym(&self, name: &str) -> Result<*mut c_void, String> {
let inner = if let Some(ref inner) = self.lib {
inner
} else {
return Err("The library has been closed".to_string());
};

unsafe {
self.lib
inner
.get(name.as_bytes())
.map(|f: libloading::Symbol<*mut c_void>| *f)
.map_err(|err| err.to_string())
}
}

pub fn is_open(&self) -> bool {
self.is_open_g.as_ref().clone()
pub fn is_closed(&self) -> bool {
self.lib.is_none()
}

pub fn close(&self) -> Result<(), libloading::Error> {
if let Err(e) = self.lib.close() {
Err(e)
} else {
mem::replace(self.is_open_g.as_mut(), false);
Ok(())
}
pub fn close(&mut self) {
drop(self.lib.take());
}
}

pub struct ExternalLibs {
pub libraries: HashMap<String, PyRef<SharedLibrary>>,
libraries: HashMap<String, PyRef<SharedLibrary>>,
}

impl ExternalLibs {
Expand All @@ -281,25 +335,24 @@ impl ExternalLibs {
library_path: &str,
vm: &VirtualMachine,
) -> Result<&PyRef<SharedLibrary>, libloading::Error> {
let library = self
.libraries
.entry(library_path.to_string())
.or_insert(SharedLibrary::new(library_path)?.into_ref(vm));

if !library.is_open() {
if let Some(l) = self.libraries.insert(
library_path.to_string(),
SharedLibrary::new(library_path)?.into_ref(vm),
) {
// Ok(self.libraries.get_mut(library_path.to_string()))
Ok(&l)
} else {
// @TODO: What this error should be?
Err(libloading::Error::DlOpenUnknown)
match self.libraries.get(&library_path.to_string()) {
Some(l) => {
if l.is_closed() {
self.libraries.insert(
library_path.to_string(),
SharedLibrary::new(library_path)?.into_ref(vm),
);
}
}
} else {
Ok(library)
}
_ => {
self.libraries.insert(
library_path.to_string(),
SharedLibrary::new(library_path)?.into_ref(vm),
);
}
};

Ok(self.libraries.get(&library_path.to_string()).unwrap())
}
}

Expand Down
28 changes: 18 additions & 10 deletions 28 vm/src/stdlib/ctypes/dll.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,18 +32,15 @@ pub fn dlsym(slib: PyObjectRc, func: PyObjectRc, vm: &VirtualMachine) -> PyResul
if !vm.isinstance(&func, &vm.ctx.types.str_type)? {
return Err(vm.new_value_error("second argument (func) must be str".to_string()));
}

let func_name = func.downcast::<PyStr>().unwrap().as_ref();
let str_ref = func.downcast::<PyStr>().unwrap();
let func_name = str_ref.as_ref();

match slib.downcast::<SharedLibrary>() {
Ok(lib) => {
if !lib.is_open() {
Err(vm.new_runtime_error(format!("Library is closed.")))
} else if let Ok(ptr) = lib.get_sym(func_name) {
Ok(PyInt::from(ptr as *const _ as usize))
} else {
// @TODO: Change this error message
Err(vm.new_runtime_error(format!("Error while opening symbol {}", func_name)))
match lib.get_sym(func_name) {
Ok(ptr) => Ok(PyInt::from(ptr as *const _ as usize)),
Err(e) => Err(vm
.new_runtime_error(format!("Error while opening symbol {}: {}", func_name, e))),
}
}
Err(_) => {
Expand All @@ -53,4 +50,15 @@ pub fn dlsym(slib: PyObjectRc, func: PyObjectRc, vm: &VirtualMachine) -> PyResul
}
}

pub fn dlclose() {}
pub fn dlclose(slib: PyObjectRc, vm: &VirtualMachine) -> PyResult {
match slib.downcast::<SharedLibrary>() {
Ok(lib) => {
lib.close();
Ok(vm.ctx.none())
}
Err(_) => {
Err(vm
.new_value_error("first argument (slib) is not a valid SharedLibrary".to_string()))
}
}
}
46 changes: 13 additions & 33 deletions 46 vm/src/stdlib/ctypes/function.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ use crate::stdlib::ctypes::common::{CDataObject, Function, SharedLibrary, SIMPLE
use crate::slots::Callable;
use crate::stdlib::ctypes::dll::dlsym;

fn map_types_to_res(args: &Vec<PyObjectRc>, vm: &VirtualMachine) -> PyResult<Vec<PyObjectRef>> {
fn map_types_to_res(args: &[PyObjectRc], vm: &VirtualMachine) -> PyResult<Vec<PyObjectRef>> {
args.iter()
.enumerate()
.map(|(idx, inner_obj)| {
Expand Down Expand Up @@ -75,14 +75,15 @@ impl PyCFuncPtr {
{
let args: Vec<PyObjectRef> = vm.extract_elements(&argtypes).unwrap();

let c_args = map_types_to_res(&args, vm);
let c_args = map_types_to_res(&args, vm)?;

self._argtypes_.write().clear();
self._argtypes_.write().extend(c_args?.into_iter());
self._argtypes_.write().extend(c_args.clone().into_iter());

let fn_ptr = self._f.write().as_mut();
let mut f_guard = self._f.write();
let fn_ptr = f_guard.as_mut();

let str_types: Result<Vec<String>, _> = c_args?
let str_types: Result<Vec<String>, _> = c_args
.iter()
.map(|obj| {
if let Ok(attr) = vm.get_attribute(obj.clone(), "_type_") {
Expand All @@ -107,17 +108,17 @@ impl PyCFuncPtr {
#[pyproperty(name = "_restype_", setter)]
fn set_restype(&self, restype: PyObjectRef, vm: &VirtualMachine) -> PyResult<()> {
match vm.isinstance(&restype, CDataObject::static_type()) {
Ok(_) => match vm.get_attribute(restype, "_type_") {
Ok(_) => match vm.get_attribute(restype.clone(), "_type_") {
Ok(_type_)
if vm.isinstance(&_type_, &vm.ctx.types.str_type)?
&& _type_.to_string().len() == 1
&& SIMPLE_TYPE_CHARS.contains(_type_.to_string().as_str()) =>
{
let dest = self._restype_.write().as_mut();
let src = restype.clone();
mem::replace(dest, src);
let mut r_guard = self._restype_.write();
mem::replace(r_guard.as_mut(), restype.clone());

let fn_ptr = self._f.write().as_mut();
let mut a_guard = self._f.write();
let fn_ptr = a_guard.as_mut();
fn_ptr.set_ret(_type_.to_string().as_str());

Ok(())
Expand Down Expand Up @@ -164,8 +165,7 @@ impl PyCFuncPtr {
if let Ok(h) = vm.get_attribute(arg.clone(), "_handle") {
if let Ok(handle) = h.downcast::<SharedLibrary>() {
let handle_obj = handle.into_object();
let ptr_fn = dlsym(handle_obj.clone(), func_name.into_object(), vm)?;

let ptr_fn = dlsym(handle_obj.clone(), func_name.clone().into_object(), vm)?;
let fn_ptr = usize::try_from_object(vm, ptr_fn.into_object(vm))? as *mut c_void;

PyCFuncPtr {
Expand Down Expand Up @@ -208,26 +208,6 @@ impl Callable for PyCFuncPtr {
// Needs to check their types and convert to middle::Arg based on zelf._argtypes_
let arg_vec = map_types_to_res(&args.args, vm)?;

// This is not optimal, but I can't simply store a vector of middle::Type inside PyCFuncPtr
let c_args: Result<Vec<String>, ()> = zelf
._argtypes_
.read()
.iter()
.map(|obj| {
if let Ok(attr) = vm.get_attribute(obj.clone(), "_type_") {
Ok(attr.to_string())
} else {
Err(())
}
})
.collect();

let ret_type = zelf._restype_.read().as_ref();

let res = zelf._f.read().call(arg_vec, vm);

if let Ok(value) = res {
} else {
}
zelf._f.write().as_mut().call(arg_vec, vm)
}
}
Loading
Morty Proxy This is a proxified and sanitized view of the page, visit original site.