c++ - Boost.Python: Wrap functions to release the GIL -
i working boost.python , solve tricky problem.
context
when c++ method/function exposed python, needs release gil (global interpreter lock) let other threads use interpreter. way, when python code calls c++ function, interpreter can used other threads. now, each c++ function looks this:
// module.cpp int myfunction(std::string question) { releasegil unlockgil; return 42; } to pass boost python, do:
// python_exposure.cpp boost_python_module(pythonmodule) { def("myfunction", &myfunction); } problem
this scheme works fine, implies module.cpp depends on boost.python no reason. ideally, python_exposure.cpp should depend on boost.python.
solution?
my idea play boost.function wrap function calls this:
// python_exposure.cpp boost_python_module(pythonmodule) { def("myfunction", wrap(&myfunction)); } here wrap in charge of unlocking gil during call myfunction. problem method wrap needs have same signature myfunction pretty mean re-implementing boost.function...
i thankful if had suggestion problem.
exposing functors methods not officially supported. supported approach expose non-member function delegates member-function. however, can result in large amount of boilerplate code.
as best can tell, boost.python's implementation not explicitly preclude functors, allows instances of python::object exposed method. however, boost.python place requirements on type of object being exposed method:
- the functor copyconstructible.
- the functor callable. i.e. instance
ocan calledo(a1, a2, a3). - the call signature must available meta-data during runtime. boost.python calls
boost::python::detail::get_signature()function obtain meta-data. meta-data used internally setup proper invocation, dispatching python c++.
the latter requirement gets complex. reason not clear me, boost.python invokes get_signature() through qualified-id, preventing argument dependent lookup. therefore, candidates get_signature() must declared before calling template's definition context. example, overloads get_signature() considered declared before definition of templates invoke it, such class_, def(), , make_function(). account behavior, when enabling functor in boost.python, 1 must provide get_signature() overload prior including boost.python or explicitly provide meta-sequence representing signature make_function().
lets work through examples of enabling functor support, providing functors support guards. have opted not use c++11 features. such, there boilerplate code reduced variadic templates. additionally, of examples use same model provides 2 non-member functions , spam class has 2 member-functions:
/// @brief mockup class member functions. class spam { public: void action() { std::cout << "spam::action()" << std::endl; } int times_two(int x) { std::cout << "spam::times_two()" << std::endl; return 2 * x; } }; // mockup non-member functions. void action() { std::cout << "action()" << std::endl; } int times_two(int x) { std::cout << "times_two()" << std::endl; return 2 * x; } enabling boost::function
when using preferred syntax boost.function, decomposing signature meta-data meets boost.python requirements can done boost.functiontypes. here complete example enabling boost::function functors exposed boost.python method:
#include <iostream> #include <boost/function.hpp> #include <boost/function_types/components.hpp> namespace boost { namespace python { namespace detail { // get_signature overloads must declared before including // boost/python.hpp. declaration must visible @ // point of definition of various boost.python templates during // first phase of 2 phase lookup. boost.python invokes // get_signature function via qualified-id, adl disabled. /// @brief signature of boost::function. template <typename signature> inline typename boost::function_types::components<signature>::type get_signature(boost::function<signature>&, void* = 0) { return typename boost::function_types::components<signature>::type(); } } // namespace detail } // namespace python } // namespace boost #include <boost/python.hpp> /// @brief mockup class member functions. class spam { public: void action() { std::cout << "spam::action()" << std::endl; } int times_two(int x) { std::cout << "spam::times_two()" << std::endl; return 2 * x; } }; // mockup non-member functions. void action() { std::cout << "action()" << std::endl; } int times_two(int x) { std::cout << "times_two()" << std::endl; return 2 * x; } boost_python_module(example) { namespace python = boost::python; // expose class , member-function. python::class_<spam>("spam") .def("action", &spam::action) .def("times_two", boost::function<int(spam&, int)>( &spam::times_two)) ; // expose non-member function. python::def("action", &action); python::def("times_two", boost::function<int()>( boost::bind(×_two, 21))); } and usage:
>>> import example >>> spam = example.spam() >>> spam.action() spam::action() >>> spam.times_two(5) spam::times_two() 10 >>> example.action() action() >>> example.times_two() times_two() 42 when providing functor invoke member-function, provided signature needs non-member function equivalent. in case, int(spam::*)(int) becomes int(spam&, int).
// ... .def("times_two", boost::function<int(spam&, int)>( &spam::times_two)) ; also, arguments can bound functors boost::bind. example, calling example.times_two() not have provide argument, 21 bound functor.
python::def("times_two", boost::function<int()>( boost::bind(×_two, 21))); custom functor guards
expanding upon above example, 1 can enable custom functor types used boost.python. lets create functor, called guarded_function, use raii, invoking wrapped function during raii object's lifetime.
/// @brief functor invoke function while holding guard. /// upon returning function, guard released. template <typename signature, typename guard> class guarded_function { public: typedef typename boost::function_types::result_type<signature>::type result_type; template <typename fn> guarded_function(fn fn) : fn_(fn) {} result_type operator()() { guard g; return fn_(); } // ... overloads operator() private: boost::function<signature> fn_; }; the guarded_function provides similar semantics python with statement. thus, keep boost.python api name choices, with() c++ function provide way create functors.
/// @brief create callable object guards. template <typename guard, typename fn> boost::python::object with(fn fn) { return boost::python::make_function( guarded_function<guard, fn>(fn), ...); } this allows functions exposed run guard in non-intrusive manner:
class no_gil; // guard // ... .def("times_two", with<no_gil>(&spam::times_two)) ; additionally, with() function provides ability deduce function signatures, allowing meta-data signature explicitly provided boost.python rather having overload boost::python::detail::get_signature().
here complete example, using 2 raii types:
no_gil: releases gil in constructor, , reacquires gil in destructor.echo_guard: prints in constructor , destructor.
#include <iostream> #include <boost/function.hpp> #include <boost/function_types/components.hpp> #include <boost/function_types/function_type.hpp> #include <boost/function_types/result_type.hpp> #include <boost/python.hpp> #include <boost/tuple/tuple.hpp> namespace detail { /// @brief functor invoke function while holding guard. /// upon returning function, guard released. template <typename signature, typename guard> class guarded_function { public: typedef typename boost::function_types::result_type<signature>::type result_type; template <typename fn> guarded_function(fn fn) : fn_(fn) {} result_type operator()() { guard g; return fn_(); } template <typename a1> result_type operator()(a1 a1) { guard g; return fn_(a1); } template <typename a1, typename a2> result_type operator()(a1 a1, a2 a2) { guard g; return fn_(a1, a2); } private: boost::function<signature> fn_; }; /// @brief provides signature type. template <typename signature> struct mpl_signature { typedef typename boost::function_types::components<signature>::type type; }; // support boost::function. template <typename signature> struct mpl_signature<boost::function<signature> >: public mpl_signature<signature> {}; /// @brief create callable object guards. template <typename guard, typename fn, typename policy> boost::python::object with_aux(fn fn, const policy& policy) { // obtain components of fn. decompose non-member // , member functions mpl sequence. // r (*)(a1) => r, a1 // r (c::*)(a1) => r, c*, a1 typedef typename mpl_signature<fn>::type mpl_signature_type; // synthesize components function type. process // causes member functions require instance argument. // necessary because member functions explicitly // provided 'self' argument. // r, a1 => r (*)(a1) // r, c*, a1 => r (*)(c*, a1) typedef typename boost::function_types::function_type< mpl_signature_type>::type signature_type; // create callable boost::python::object delegates // guarded_function. return boost::python::make_function( guarded_function<signature_type, guard>(fn), policy, mpl_signature_type()); } } // namespace detail /// @brief create callable object guards. template <typename guard, typename fn, typename policy> boost::python::object with(const fn& fn, const policy& policy) { return detail::with_aux<guard>(fn, policy); } /// @brief create callable object guards. template <typename guard, typename fn> boost::python::object with(const fn& fn) { return with<guard>(fn, boost::python::default_call_policies()); } /// @brief mockup class member functions. class spam { public: void action() { std::cout << "spam::action()" << std::endl; } int times_two(int x) { std::cout << "spam::times_two()" << std::endl; return 2 * x; } }; // mockup non-member functions. void action() { std::cout << "action()" << std::endl; } int times_two(int x) { std::cout << "times_two()" << std::endl; return 2 * x; } /// @brief guard unlock gil upon construction, , /// reacquire upon destruction. struct no_gil { public: no_gil() { state_ = pyeval_savethread(); std::cout << "no_gil()" << std::endl; } ~no_gil() { std::cout << "~no_gil()" << std::endl; pyeval_restorethread(state_); } private: pythreadstate* state_; }; /// @brief guard prints std::cout. struct echo_guard { echo_guard() { std::cout << "echo_guard()" << std::endl; } ~echo_guard() { std::cout << "~echo_guard()" << std::endl; } }; boost_python_module(example) { namespace python = boost::python; // expose class , member-function. python::class_<spam>("spam") .def("action", &spam::action) .def("times_two", with<no_gil>(&spam::times_two)) ; // expose non-member function. python::def("action", &action); python::def("times_two", with<boost::tuple<no_gil, echo_guard> >( ×_two)); } and usage:
>>> import example >>> spam = example.spam() >>> spam.action() spam::action() >>> spam.times_two(5) no_gil() spam::times_two() ~no_gil() 10 >>> example.action() action() >>> example.times_two(21) no_gil() echo_guard() times_two() ~echo_guard() ~no_gil() 42 notice how multiple guards can provided using container type, such boost::tuple:
python::def("times_two", with<boost::tuple<no_gil, echo_guard> >( ×_two)); when invoked in python, example.times_two(21) produces following output:
no_gil() echo_guard() times_two() ~echo_guard() ~no_gil() 42
Comments
Post a Comment