PyInit already defined – PyBind

I want to build a module for python, and it compiled fine when I had 1 header, 1 cpp file, but I wanted to extend it to 1 header, 2 cpp files, and now I get errors I don’t really understand.

My Header (fast_qs_module.h):

#ifndef FASTCOSINE_MODULE_INCLUDE_H
#define FASTCOSINE_MODULE_INCLUDE_H

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <Python.h>
#include <vector>

std::vector<double> get_dwds_distances(std::vector<std::vector<double>> unlabeled, std::vector<std::vector<double>> labeled, double alpha);
std::vector<double> get_euclidian_distances_to_reference(std::vector<std::vector<double>> unlabeled, std::vector<double> reference);

PYBIND11_MODULE(fast_cosine, mod) {
    mod.def("get_dwds_distances", &get_dwds_distances, "Multithreaded DWDS calculations to get the values for each unlabeled sample.");
    mod.def("get_euclidian_distances", &get_euclidian_distances_to_reference, "Multithreaded Euclidian distance calculations to get the values for each unlabeled sample.");
}

#endif  // FASTCOSINE_MODULE_INCLUDE_H

One cpp file:

#include [...]

#include "fast_qs_module.h"

class EuclidianClass
{
    [...]
};

std::vector<double> get_euclidian_distances_to_reference(std::vector<std::vector<double>> unlabeled, std::vector<double> reference)
{
    EuclidianClass runner(unlabeled, reference);
    return runner.get_euclidian_result();
}

The other cpp file:

#include [...]

#include "fast_qs_module.h"

class CosineClass
{
     [...]
};

std::vector<double> get_dwds_distances(std::vector<std::vector<double>> unlabeled, std::vector<std::vector<double>> labeled, double alpha)
{
    CosineClass runner(unlabeled, labeled, alpha);
    return runner.get_dwds_result();
}

When I pack it all in one cpp it works, but why doesn’t it work with two cpp files? The header is include guarded…

The error I am getting:

 compiler_compat/ld: build/temp.linux-x86_64-cpython-38/src/euclidian.o: in function `PyInit_fast_cosine':
    euclidian.cpp:(.text+0x810): multiple definition of `PyInit_fast_cosine'; build/temp.linux-x86_64-cpython-38/src/cdist_cosine.o:cdist_cosine.cpp:(.text+0xfc0): first defined here
    collect2: error: ld returned 1 exit status

As per https://pybind11.readthedocs.io/en/stable/faq.html#how-can-i-reduce-the-build-time I split it like this:

Header:

void init_dwds(py::module_ &);
void init_euclidian(py::module_ &);

PYBIND11_MODULE(fast_cosine, mod) {
    init_dwds(mod);
    init_euclidian(mod);
}

Source files:

std::vector<double> get_dwds_distances(std::vector<std::vector<double>> unlabeled, std::vector<std::vector<double>> labeled, double alpha)
{
    CosineClass runner(unlabeled, labeled, alpha);
    return runner.get_dwds_result();
}

void init_dwds(py::module_ &mod)
{
    mod.def("get_dwds_distances", &get_dwds_distances, "Multithreaded DWDS calculations to get the values for each unlabeled sample.");
}

Source 2:

std::vector<double> get_euclidian_distances_to_reference(std::vector<std::vector<double>> unlabeled, std::vector<double> reference)
{
    EuclidianClass runner(unlabeled, reference);
    return runner.get_euclidian_result();
}

void init_euclidian(py::module_ &mod)
{
    mod.def("get_euclidian_distances", &get_euclidian_distances_to_reference, "Multithreaded Euclidian distance calculations to get the values for each unlabeled sample.");
}

but I still get

euclidian.cpp:(.text+0x810): multiple definition of `PyInit_fast_cosine'; build/temp.linux-x86_64-cpython-38/src/cdist_cosine.o:cdist_cosine.cpp:(.text+0xfc0): first defined here
    collect2: error: ld returned 1 exit status

>Solution :

Include guard only protects a header to be included more than once in a single translation unit. It does not help if a header contains something that needs not be placed in the header.

In your case, it looks like PYBIND11_MODULE expands to something like a definition rather than a declaration, which should not be in a header. Placing it in a cpp should solve.