LSTM model on python, reshape of the prediction set

I’m working on a LSTM model for a timeseries prediction.
It works well until I have to reshape the output of the model in order to compute an accuracy index and plot the results. This is my code:

train, test = btc.loc[btc.index <= '2020-12-31'], btc.loc[btc.index > '2020-12-31']

scaler = StandardScaler()
scaler = scaler.fit(train)

train, test = scaler.transform(train), scaler.transform(test)

def to_sequences(x, y, seq_size=1):
    x_values = []
    y_values = []

    for i in range(len(x)-seq_size):
        
        x_values.append(x[i:(i+seq_size)])
        y_values.append(y[i+seq_size])
        
    return np.array(x_values), np.array(y_values)

seq_size = 30

xtrain, ytrain = to_sequences(train, train, seq_size)
xtest, ytest = to_sequences(test, test, seq_size)

model = Sequential()
model.add(LSTM(128, input_shape=(xtrain.shape[1], xtrain.shape[2])))
model.add(Dropout(rate=0.2))
model.add(RepeatVector(xtrain.shape[1]))
model.add(LSTM(128, return_sequences=True))
model.add(Dropout(rate=0.2))
model.add(TimeDistributed(Dense(xtrain.shape[2])))
model.compile(optimizer='adam', loss='mae')
model.summary()

# fit model
history = model.fit(xtrain, ytrain, epochs=10, batch_size=32, validation_split=0.1, verbose=1)

trainPredict = model.predict(xtrain)
testPredict = model.predict(xtest)

After the model prediction, I don’t know how to manage it, because the shape of the prediction is (1023,30,1) but I obviously need a reshape in order to have a 2D array. If I reshape with .reshape(-1,1) I obtain a 2D array with (1023, 30) but I need a 2D array with (1203, 1), i.e., an array with only the predicted values.

>Solution :

The output of the model prediction should already have shape (1203, 1). You shouldn’t need to mangle it further to get to the right shape.

You are using Dense layer from TimeDistributed which uses the same dense layer for each time dimension. If you want the final prediction, you should just use the last time dimension and you could do that by changing return_sequences=True to False in your last LSTM layer and use a regular Dense layer as output layer.