Why PyList_Append is called each time a list is evaluated?

December 1, 2022

I’m working with CPython3.11.0a3+. I added a break point at PyList_Append and modified the function to stop when the newitem is a dict. The original function:

int
PyList_Append(PyObject *op, PyObject *newitem)
{
    if (PyList_Check(op) && (newitem != NULL))
        return app1((PyListObject *)op, newitem);
    PyErr_BadInternalCall();
    return -1;
}

Modified version:

int
PyList_Append(PyObject *op, PyObject *newitem)
{
    if (PyDict_CheckExact(newitem)) {
        fprintf(stdout, "hello\n");
    }
    if (PyList_Check(op) && (newitem != NULL))
        return app1((PyListObject *)op, newitem);
    PyErr_BadInternalCall();
    return -1;
}

I placed a break point on line fprintf(stdout, "hello\n");. This is because in python startup process, there are infinite calls of PyList_Append but none of them are with newitem of type dict. So in this way I can escape all of those calls and reach the repl to workaround with my own variables.

(gdb) break Objects/listobject.c:328
Breakpoint 1 at 0x44fb00: file Objects/listobject.c, line 328.
(gdb) run
Starting program: /home/amirreza/Desktop/edu/python_internals/1/cpython/python 
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.31-2.fc32.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Python 3.11.0a3+ (heads/main-dirty:1cbb887, Dec  1 2022, 12:22:29) [GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = []
>>> a.append(3)
>>> a
[3]
>>> a.append({})
>>> 
>>> a

Breakpoint 1, PyList_Append (op=op@entry=0x7fffea8e9780, newitem=newitem@entry=0x7fffeaa22bc0) at Objects/listobject.c:328
328         fprintf(stdout, "hello\n");
Missing separate debuginfos, use: dnf debuginfo-install ncurses-libs-6.1-15.20191109.fc32.x86_64 readline-8.0-4.fc32.x86_64
(gdb) c
Continuing.
hello
[3, {}]
>>> 
>>> a

Breakpoint 1, PyList_Append (op=op@entry=0x7fffea8e9780, newitem=newitem@entry=0x7fffeaa22bc0) at Objects/listobject.c:328
328         fprintf(stdout, "hello\n");
(gdb) c
Continuing.
hello
[3, {}]
>>> 
>>> a

Breakpoint 1, PyList_Append (op=op@entry=0x7fffea8e9780, newitem=newitem@entry=0x7fffeaa22bc0) at Objects/listobject.c:328
328         fprintf(stdout, "hello\n");
(gdb) c
Continuing.
hello
[3, {}]
>>>

My question is why each time I only try to evaluate the a list, the PyList_Append is invoked? For the first time we can ignore it due to lazy-evaluation. But for the second and third time, why it’s called every time?

>Solution :

dict.__repr__ uses the CPython-internal Py_ReprEnter and Py_ReprLeave functions to stop infinite recursion for recursively nested data structures, and Py_ReprEnter appends the dict to a list of objects currently having their repr evaluated in the running thread.