ValueError: probabilities do not sum to 1

May 7, 2023

Don’t understand why am getting the following value error probabilities do not sum to 1. Is this due to my joint distribution? a numpy error? any help greatly appreciated.

import numpy as np
import matplotlib.pyplot as plt

p = np.array([[10/66, 15/66, 3/66], [20/66, 12/66, 0], [6/66, 0, 0]])

# Initialize starting values for X and Y
x = 0
y = 0

# Create arrays to store trace plots
trace_x = []
trace_y = []

# Run 10,000 iterations of Gibbs sampling
for i in range(10000):
    # Sample a new value for X given the current value of Y
    x = np.random.choice([0, 1, 2], p=p[y])
    # Sample a new value for Y given the current value of X
    y = np.random.choice([0, 1, 2], p=p[:, x])
    # Append the new values to the trace plots
    trace_x.append(x)
    trace_y.append(y)

# Plot the trace plots
plt.plot(trace_x)
plt.title('Trace plot for X')
plt.xlabel('Iteration')
plt.ylabel('Value')
plt.show()

plt.plot(trace_y)
plt.title('Trace plot for Y')
plt.xlabel('Iteration')
plt.ylabel('Value')
plt.show()

>Solution :

You’re providing P(X, Y=y) rather than P(X | Y=y). There’s a scaling term missing because P(X | Y=y) = P(X, Y=y) / P(Y=y). In other words the rows and columns of the joint distribution shouldn’t add to one. You need to normalize them yourself to get the conditional distributions.

# Sample a new value for X given the current value of Y
py = np.sum(p[y])
x = np.random.choice([0, 1, 2], p=p[y] / py)
# Sample a new value for Y given the current value of X
px = np.sum(p[:, x])
y = np.random.choice([0, 1, 2], p=p[:, x] / px)