Commit b2c2c792 authored by Wenzel Jakob's avatar Wenzel Jakob
Browse files

improved handling of shared/smart pointers

Previously, pybind11 required classes using std::shared_ptr<> to derive
from std::enable_shared_from_this<> (or compilation failures would ensue).

Everything now also works for classes that don't do this, assuming that
some basic rules are followed (e.g. never passing "raw" pointers of
instances manged by shared pointers). The safer
std::enable_shared_from_this<> approach continues to be supported.
parent 2ca07de8
......@@ -521,13 +521,9 @@ following snippet causes ``std::shared_ptr`` to be used instead.
.. code-block:: cpp
/// Type declaration
class Example : public std::enable_shared_from_this<Example> /* <- important, see below */ {
// ...
};
py::class_<Example, std::shared_ptr<Example> /* <- holder type */> obj(m, "Example");
/// .... code within PYBIND11_PLUGIN declaration .....
py::class_<Example, std::shared_ptr<Example> /* <- important */> obj(m, "Example");
Note that any particular class can only be associated with a single holder type.
To enable transparent conversions for functions that take shared pointers as an
argument or that return them, a macro invocation similar to the following must
......@@ -537,7 +533,7 @@ be declared at the top level before any binding code:
PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
.. warning::
.. note::
The first argument of :func:`PYBIND11_DECLARE_HOLDER_TYPE` should be a
placeholder name that is used as a template parameter of the second
......@@ -545,25 +541,74 @@ be declared at the top level before any binding code:
both sides; also, don't use the name of a type that already exists in your
codebase.
.. warning::
One potential stumbling block when using holder types is that they need to be
applied consistently. Can you guess what's broken about the following binding
code?
.. code-block:: cpp
class Child { };
To ensure correct reference counting among Python and C++, the use of
``std::shared_ptr<T>`` as a holder type requires that ``T`` inherits from
``std::enable_shared_from_this<T>`` (see cppreference_ for details).
class Parent {
public:
Parent() : child(std::make_shared<Child>()) { }
Child *get_child() { return child.get(); } /* Hint: ** DON'T DO THIS ** */
private:
std::shared_ptr<Child> child;
};
If you encounter issues (failure to compile, ``bad_weak_ptr`` exceptions),
please check that you really did all three steps:
PYBIND11_PLUGIN(example) {
py::module m("example");
1. invoking the ``PYBIND11_DECLARE_HOLDER_TYPE`` macro in every file that
contains pybind11 code and uses your chosen smart pointer type.
py::class_<Child, std::shared_ptr<Child>>(m, "Child");
2. specifying the holder types to ``class_``.
py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent")
.def(py::init<>())
.def("get_child", &Parent::get_child);
return m.ptr();
}
3. extending from ``std::enable_shared_from_this`` when using
``std::shared_ptr``.
The following Python code will cause undefined behavior (and likely a
segmentation fault).
.. code-block:: python
from example import Parent
print(Parent().get_child())
The problem is that ``Parent::get_child()`` returns a pointer to an instance of
``Child``, but the fact that this instance is already managed by
``std::shared_ptr<...>`` is lost when passing raw pointers. In this case,
pybind11 will create a second independent ``std::shared_ptr<...>`` that also
claims ownership of the pointer. In the end, the object will be freed **twice**
since these shared pointers have no way of knowing about each other.
There are two ways to resolve this issue:
1. For types that are managed by a smart pointer class, never use raw pointers
in function arguments or return values. In other words: always consistently
wrap pointers into their designated holder types (such as
``std::shared_ptr<...>``). In this case, the signature of ``get_child()``
should be modified as follows:
.. code-block:: cpp
std::shared_ptr<Child> get_child() { return child; }
2. Adjust the definition of ``Child`` by specifying
``std::enable_shared_from_this<T>`` (see cppreference_ for details) as a
base class. This adds a small bit of information to ``Child`` that allows
pybind11 to realize that there is already an existing
``std::shared_ptr<...>`` and communicate with it. In this case, the
declaration of ``Child`` should look as follows:
.. _cppreference: http://en.cppreference.com/w/cpp/memory/enable_shared_from_this
.. code-block:: cpp
class Child : public std::enable_shared_from_this<Child> { };
.. seealso::
The file :file:`example/example8.cpp` contains a complete example that
......
......@@ -11,19 +11,19 @@
#include "example.h"
#include "object.h"
/// Object subclass
class MyObject : public Object {
/// Custom object with builtin reference counting (see 'object.h' for the implementation)
class MyObject1 : public Object {
public:
MyObject(int value) : value(value) {
MyObject1(int value) : value(value) {
std::cout << toString() << " constructor" << std::endl;
}
std::string toString() const {
return "MyObject[" + std::to_string(value) + "]";
return "MyObject1[" + std::to_string(value) + "]";
}
protected:
virtual ~MyObject() {
virtual ~MyObject1() {
std::cout << toString() << " destructor" << std::endl;
}
......@@ -31,7 +31,8 @@ private:
int value;
};
class MyObject2 : public std::enable_shared_from_this<MyObject2> {
/// Object managed by a std::shared_ptr<>
class MyObject2 {
public:
MyObject2(int value) : value(value) {
std::cout << toString() << " constructor" << std::endl;
......@@ -49,52 +50,80 @@ private:
int value;
};
/// Make pybind aware of the ref-counted wrapper type
/// Object managed by a std::shared_ptr<>, additionally derives from std::enable_shared_from_this<>
class MyObject3 : public std::enable_shared_from_this<MyObject3> {
public:
MyObject3(int value) : value(value) {
std::cout << toString() << " constructor" << std::endl;
}
std::string toString() const {
return "MyObject3[" + std::to_string(value) + "]";
}
virtual ~MyObject3() {
std::cout << toString() << " destructor" << std::endl;
}
private:
int value;
};
/// Make pybind aware of the ref-counted wrapper type (s)
PYBIND11_DECLARE_HOLDER_TYPE(T, ref<T>);
PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
Object *make_object_1() { return new MyObject(1); }
ref<Object> make_object_2() { return new MyObject(2); }
MyObject *make_myobject_4() { return new MyObject(4); }
ref<MyObject> make_myobject_5() { return new MyObject(5); }
Object *make_object_1() { return new MyObject1(1); }
ref<Object> make_object_2() { return new MyObject1(2); }
MyObject2 *make_myobject2_1() { return new MyObject2(1); }
std::shared_ptr<MyObject2> make_myobject2_2() { return std::make_shared<MyObject2>(2); }
MyObject1 *make_myobject1_1() { return new MyObject1(4); }
ref<MyObject1> make_myobject1_2() { return new MyObject1(5); }
MyObject2 *make_myobject2_1() { return new MyObject2(6); }
std::shared_ptr<MyObject2> make_myobject2_2() { return std::make_shared<MyObject2>(7); }
MyObject3 *make_myobject3_1() { return new MyObject3(8); }
std::shared_ptr<MyObject3> make_myobject3_2() { return std::make_shared<MyObject3>(9); }
void print_object_1(const Object *obj) { std::cout << obj->toString() << std::endl; }
void print_object_2(ref<Object> obj) { std::cout << obj->toString() << std::endl; }
void print_object_3(const ref<Object> &obj) { std::cout << obj->toString() << std::endl; }
void print_object_4(const ref<Object> *obj) { std::cout << (*obj)->toString() << std::endl; }
void print_myobject_1(const MyObject *obj) { std::cout << obj->toString() << std::endl; }
void print_myobject_2(ref<MyObject> obj) { std::cout << obj->toString() << std::endl; }
void print_myobject_3(const ref<MyObject> &obj) { std::cout << obj->toString() << std::endl; }
void print_myobject_4(const ref<MyObject> *obj) { std::cout << (*obj)->toString() << std::endl; }
void print_myobject1_1(const MyObject1 *obj) { std::cout << obj->toString() << std::endl; }
void print_myobject1_2(ref<MyObject1> obj) { std::cout << obj->toString() << std::endl; }
void print_myobject1_3(const ref<MyObject1> &obj) { std::cout << obj->toString() << std::endl; }
void print_myobject1_4(const ref<MyObject1> *obj) { std::cout << (*obj)->toString() << std::endl; }
void print_myobject2_1(const MyObject2 *obj) { std::cout << obj->toString() << std::endl; }
void print_myobject2_2(std::shared_ptr<MyObject2> obj) { std::cout << obj->toString() << std::endl; }
void print_myobject2_3(const std::shared_ptr<MyObject2> &obj) { std::cout << obj->toString() << std::endl; }
void print_myobject2_4(const std::shared_ptr<MyObject2> *obj) { std::cout << (*obj)->toString() << std::endl; }
void print_myobject3_1(const MyObject3 *obj) { std::cout << obj->toString() << std::endl; }
void print_myobject3_2(std::shared_ptr<MyObject3> obj) { std::cout << obj->toString() << std::endl; }
void print_myobject3_3(const std::shared_ptr<MyObject3> &obj) { std::cout << obj->toString() << std::endl; }
void print_myobject3_4(const std::shared_ptr<MyObject3> *obj) { std::cout << (*obj)->toString() << std::endl; }
void init_ex8(py::module &m) {
py::class_<Object, ref<Object>> obj(m, "Object");
obj.def("getRefCount", &Object::getRefCount);
py::class_<MyObject, ref<MyObject>>(m, "MyObject", obj)
py::class_<MyObject1, ref<MyObject1>>(m, "MyObject1", obj)
.def(py::init<int>());
m.def("make_object_1", &make_object_1);
m.def("make_object_2", &make_object_2);
m.def("make_myobject_4", &make_myobject_4);
m.def("make_myobject_5", &make_myobject_5);
m.def("make_myobject1_1", &make_myobject1_1);
m.def("make_myobject1_2", &make_myobject1_2);
m.def("print_object_1", &print_object_1);
m.def("print_object_2", &print_object_2);
m.def("print_object_3", &print_object_3);
m.def("print_object_4", &print_object_4);
m.def("print_myobject_1", &print_myobject_1);
m.def("print_myobject_2", &print_myobject_2);
m.def("print_myobject_3", &print_myobject_3);
m.def("print_myobject_4", &print_myobject_4);
m.def("print_myobject1_1", &print_myobject1_1);
m.def("print_myobject1_2", &print_myobject1_2);
m.def("print_myobject1_3", &print_myobject1_3);
m.def("print_myobject1_4", &print_myobject1_4);
py::class_<MyObject2, std::shared_ptr<MyObject2>>(m, "MyObject2")
.def(py::init<int>());
......@@ -105,5 +134,14 @@ void init_ex8(py::module &m) {
m.def("print_myobject2_3", &print_myobject2_3);
m.def("print_myobject2_4", &print_myobject2_4);
py::implicitly_convertible<py::int_, MyObject>();
py::class_<MyObject3, std::shared_ptr<MyObject3>>(m, "MyObject3")
.def(py::init<int>());
m.def("make_myobject3_1", &make_myobject3_1);
m.def("make_myobject3_2", &make_myobject3_2);
m.def("print_myobject3_1", &print_myobject3_1);
m.def("print_myobject3_2", &print_myobject3_2);
m.def("print_myobject3_3", &print_myobject3_3);
m.def("print_myobject3_4", &print_myobject3_4);
py::implicitly_convertible<py::int_, MyObject1>();
}
......@@ -3,49 +3,68 @@ from __future__ import print_function
import sys
sys.path.append('.')
from example import MyObject
from example import MyObject1
from example import MyObject2
from example import MyObject3
from example import make_object_1
from example import make_object_2
from example import make_myobject_4
from example import make_myobject_5
from example import make_myobject1_1
from example import make_myobject1_2
from example import make_myobject2_1
from example import make_myobject2_2
from example import make_myobject3_1
from example import make_myobject3_2
from example import print_object_1
from example import print_object_2
from example import print_object_3
from example import print_object_4
from example import print_myobject_1
from example import print_myobject_2
from example import print_myobject_3
from example import print_myobject_4
from example import print_myobject1_1
from example import print_myobject1_2
from example import print_myobject1_3
from example import print_myobject1_4
from example import print_myobject2_1
from example import print_myobject2_2
from example import print_myobject2_3
from example import print_myobject2_4
for o in [make_object_1(), make_object_2(), MyObject(3)]:
from example import print_myobject3_1
from example import print_myobject3_2
from example import print_myobject3_3
from example import print_myobject3_4
for o in [make_object_1(), make_object_2(), MyObject1(3)]:
print("Reference count = %i" % o.getRefCount())
print_object_1(o)
print_object_2(o)
print_object_3(o)
print_object_4(o)
for o in [make_myobject_4(), make_myobject_5(), MyObject(6), 7]:
for o in [make_myobject1_1(), make_myobject1_2(), MyObject1(6), 7]:
print(o)
if not isinstance(o, int):
print_object_1(o)
print_object_2(o)
print_object_3(o)
print_object_4(o)
print_myobject_1(o)
print_myobject_2(o)
print_myobject_3(o)
print_myobject_4(o)
print_myobject1_1(o)
print_myobject1_2(o)
print_myobject1_3(o)
print_myobject1_4(o)
for o in [make_myobject2_1(), make_myobject2_2()]:
for o in [MyObject2(8), make_myobject2_1(), make_myobject2_2()]:
print(o)
print_myobject2_1(o)
print_myobject2_2(o)
print_myobject2_3(o)
print_myobject2_4(o)
for o in [MyObject3(9), make_myobject3_1(), make_myobject3_2()]:
print(o)
print_myobject3_1(o)
print_myobject3_2(o)
print_myobject3_3(o)
print_myobject3_4(o)
MyObject[1] constructor
Initialized ref from pointer 0x7ffdb2c16a50
MyObject[2] constructor
Initialized ref from pointer 0x7ffdb2c167a0
Initialized ref from pointer 0x7ffdb2c167a0
Destructing ref 0x7ffdb2c167a0
MyObject[3] constructor
Initialized ref from pointer 0x7ffdb2c17420
MyObject1[1] constructor
Initialized ref from pointer 0x1347ba0
MyObject1[2] constructor
Initialized ref from pointer 0x12b9270
Initialized ref from ref 0x12b9270
Destructing ref 0x12b9270
MyObject1[3] constructor
Initialized ref from pointer 0x12a2a90
MyObject1[1]
Created empty ref
Assigning ref 0x1347ba0
Initialized ref from ref 0x1347ba0
MyObject1[1]
Destructing ref 0x1347ba0
Destructing ref 0x1347ba0
Created empty ref
Assigning ref 0x1347ba0
MyObject1[1]
Destructing ref 0x1347ba0
Created empty ref
Assigning ref 0x1347ba0
MyObject1[1]
Destructing ref 0x1347ba0
MyObject1[2]
Created empty ref
Assigning ref 0x12b9270
Initialized ref from ref 0x12b9270
MyObject1[2]
Destructing ref 0x12b9270
Destructing ref 0x12b9270
Created empty ref
Assigning ref 0x12b9270
MyObject1[2]
Destructing ref 0x12b9270
Created empty ref
Assigning ref 0x12b9270
MyObject1[2]
Destructing ref 0x12b9270
MyObject1[3]
Created empty ref
Assigning ref 0x12a2a90
Initialized ref from ref 0x12a2a90
MyObject1[3]
Destructing ref 0x12a2a90
Destructing ref 0x12a2a90
Created empty ref
Assigning ref 0x12a2a90
MyObject1[3]
Destructing ref 0x12a2a90
Created empty ref
Assigning ref 0x12a2a90
MyObject1[3]
Destructing ref 0x12a2a90
Destructing ref 0x12b9270
MyObject1[2] destructor
Destructing ref 0x1347ba0
MyObject1[1] destructor
MyObject1[4] constructor
Initialized ref from pointer 0x1347ba0
MyObject1[5] constructor
Initialized ref from pointer 0x1299190
Initialized ref from ref 0x1299190
Destructing ref 0x1299190
MyObject1[6] constructor
Initialized ref from pointer 0x133e2f0
Destructing ref 0x12a2a90
MyObject1[3] destructor
MyObject1[4]
Created empty ref
Assigning ref 0x1347ba0
Initialized ref from ref 0x1347ba0
MyObject1[4]
Destructing ref 0x1347ba0
Destructing ref 0x1347ba0
Created empty ref
Assigning ref 0x1347ba0
MyObject1[4]
Destructing ref 0x1347ba0
Created empty ref
Assigning ref 0x1347ba0
MyObject1[4]
Destructing ref 0x1347ba0
MyObject1[4]
Created empty ref
Assigning ref 0x1347ba0
Initialized ref from ref 0x1347ba0
MyObject1[4]
Destructing ref 0x1347ba0
Destructing ref 0x1347ba0
Created empty ref
Assigning ref 0x1347ba0
MyObject1[4]
Destructing ref 0x1347ba0
Created empty ref
Assigning ref 0x1347ba0
MyObject1[4]
Destructing ref 0x1347ba0
MyObject1[5]
Created empty ref
Assigning ref 0x1299190
Initialized ref from ref 0x1299190
MyObject1[5]
Destructing ref 0x1299190
Destructing ref 0x1299190
Created empty ref
Assigning ref 0x1299190
MyObject1[5]
Destructing ref 0x1299190
Created empty ref
Assigning ref 0x1299190
MyObject1[5]
Destructing ref 0x1299190
MyObject1[5]
Created empty ref
Assigning ref 0x1299190
Initialized ref from ref 0x1299190
MyObject1[5]
Destructing ref 0x1299190
Destructing ref 0x1299190
Created empty ref
Assigning ref 0x1299190
MyObject1[5]
Destructing ref 0x1299190
Created empty ref
Assigning ref 0x1299190
MyObject1[5]
Destructing ref 0x1299190
MyObject1[6]
Created empty ref
Assigning ref 0x133e2f0
Initialized ref from ref 0x133e2f0
MyObject1[6]
Destructing ref 0x133e2f0
Destructing ref 0x133e2f0
Created empty ref
Assigning ref 0x133e2f0
MyObject1[6]
Destructing ref 0x133e2f0
Created empty ref
Assigning ref 0x133e2f0
MyObject1[6]
Destructing ref 0x133e2f0
MyObject1[6]
Created empty ref
Assigning ref 0x133e2f0
Initialized ref from ref 0x133e2f0
MyObject1[6]
Destructing ref 0x133e2f0
Destructing ref 0x133e2f0
Created empty ref
Assigning ref 0x133e2f0
MyObject1[6]
Destructing ref 0x133e2f0
Created empty ref
Assigning ref 0x133e2f0
MyObject1[6]
Destructing ref 0x133e2f0
MyObject1[7] constructor
Initialized ref from pointer 0x133f3a0
MyObject1[7]
Destructing ref 0x133f3a0
MyObject1[7] destructor
Created empty ref
MyObject1[7] constructor
Initialized ref from pointer 0x12a2a90
Assigning ref 0x12a2a90
Initialized ref from ref 0x12a2a90
MyObject1[7]
Destructing ref 0x12a2a90
Destructing ref 0x12a2a90
Destructing ref 0x12a2a90
MyObject1[7] destructor
Created empty ref
MyObject1[7] constructor
Initialized ref from pointer 0x133f3a0
Assigning ref 0x133f3a0
MyObject1[7]
Destructing ref 0x133f3a0
Destructing ref 0x133f3a0
MyObject1[7] destructor
Created empty ref
MyObject1[7] constructor
Initialized ref from pointer 0x12a2a90
Assigning ref 0x12a2a90
MyObject1[7]
Destructing ref 0x12a2a90
Destructing ref 0x12a2a90
MyObject1[7] destructor
Destructing ref 0x133e2f0
MyObject1[6] destructor
Destructing ref 0x1299190
MyObject1[5] destructor
Destructing ref 0x1347ba0
MyObject1[4] destructor
MyObject2[8] constructor
MyObject2[6] constructor
MyObject2[7] constructor
MyObject2[8]
MyObject2[8]
MyObject2[8]
MyObject2[8]
MyObject2[6]
MyObject2[6]
MyObject2[6]
MyObject2[6]
MyObject2[7]
MyObject2[7]
MyObject2[7]
MyObject2[7]
MyObject2[6] destructor
MyObject2[8] destructor
MyObject3[9] constructor
MyObject3[8] constructor
MyObject3[9] constructor
MyObject2[7] destructor
MyObject3[9]
MyObject3[9]
MyObject3[9]
MyObject3[9]
MyObject3[8]
MyObject3[8]
MyObject3[8]
MyObject3[8]
MyObject3[9]
MyObject3[9]
MyObject3[9]
MyObject3[9]
MyObject3[8] destructor
MyObject3[9] destructor
Reference count = 1
MyObject[1]
Created empty ref
Initialized ref from pointer 0x7ffdb2c16a50
Move-assigning ref 0x7ffdb2c16a50
Destructing ref 0x0
Initialized ref from ref 0x7ffdb2c16a50
MyObject[1]
Destructing ref 0x7ffdb2c16a50
Destructing ref 0x7ffdb2c16a50
Created empty ref
Initialized ref from pointer 0x7ffdb2c16a50
Move-assigning ref 0x7ffdb2c16a50
Destructing ref 0x0
MyObject[1]
Destructing ref 0x7ffdb2c16a50
Created empty ref
Initialized ref from pointer 0x7ffdb2c16a50
Move-assigning ref 0x7ffdb2c16a50
Destructing ref 0x0
MyObject[1]
Destructing ref 0x7ffdb2c16a50
Reference count = 1
MyObject[2]
Created empty ref
Initialized ref from pointer 0x7ffdb2c167a0
Move-assigning ref 0x7ffdb2c167a0
Destructing ref 0x0
Initialized ref from ref 0x7ffdb2c167a0
MyObject[2]
Destructing ref 0x7ffdb2c167a0
Destructing ref 0x7ffdb2c167a0
Created empty ref
Initialized ref from pointer 0x7ffdb2c167a0
Move-assigning ref 0x7ffdb2c167a0
Destructing ref 0x0
MyObject[2]
Destructing ref 0x7ffdb2c167a0
Created empty ref
Initialized ref from pointer 0x7ffdb2c167a0
Move-assigning ref 0x7ffdb2c167a0
Destructing ref 0x0
MyObject[2]
Destructing ref 0x7ffdb2c167a0
Reference count = 1
MyObject[3]
Created empty ref
Initialized ref from pointer 0x7ffdb2c17420
Move-assigning ref 0x7ffdb2c17420
Destructing ref 0x0
Initialized ref from ref 0x7ffdb2c17420
MyObject[3]