Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

A memory leak in a simple Python C-extension

I have some code similar to the one below. That code leaks, and I don’t know why. The thing that leaks is a simple creation of a Python class’ instance inside a C code. The function I use to check the leak is create_n_times that’s defined below and just creates new Python instances and derefrences them in a loop.

This is not an MWE per-se, but part of an example. To make it easier to understand, what the code does is:

  1. The Python code defines the dataclass and registers it into the C-extension using set_ip_settings_type.
  2. Then, a C-extension function create_n_times is called and that function creates and destroys n instances of the Python dataclass.

Can anyone help?

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

In Python:

import c_api

@dataclass
class IpSettings:
    ip: str
    port: int
    dhcp: bool

c_api.set_ip_settings_type(IpSettings)
c_api.generate_n_times(100000)

In C++ I have the following code that’s compiled into a Python extension called c_api (it’s a part of that library’s definition):

#include <Python.h>

// ... Other functions including a "PyInit" function

extern "C" {
    
PyObject* ip_settings_type = NULL;
PyObject* set_ip_settings_type(PyObject* tp)
{
    Py_XDECREF(ip_settings_type);
    Py_INCREF(tp);
    ip_settings_type = tp;
    return Py_None;
}

PyObject* create_n_times(PyObject* n)
{
   long n_ = PyLong_AsLong(n);
   for (int i = 0; i < n_ ++i)
   {
       PyObject* factory_object = ip_settings_type;

       PyObject* args = PyTuple_New(3);
       PyTuple_SetItem(args, 0, PyUnicode_FromString("123.123.123.123"));
       PyTuple_SetItem(args, 1, PyLong_FromUnsignedLong(1231));
       PyTuple_SetItem(args, 2, Py_False);

       PyObject* obj = PyObject_CallObject(factory_object, args);
       Py_DECREF(obj);
   }

    return Py_None;
}

}

>Solution :

PyTuple_SetItem steals the reference to the supplied object, but Py_False is a single object. When the args tuple is destroyed, the reference count for Py_False isgetting mangled.

Use PyBool_FromLong(0) to create a new reference to Py_False, like the other two calls to PyTuple_SetItem. (see docs.python.org/3/c-api/bool.html)

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading