# How can I normalize after masking?

Suppose I have a tensor like [0.6, 0.7, 0.4] and a mask like: [1, 0,0] How can I normalize it to: [1,0,0] my try: normalized_attn_scores = F.softmax(attn_scores, 1) normalized_attn_scores = normalized_attn_scores.mul(attn_mask) But it doesn’t produce the desired output >Solution : You can normalize after masking by dividing the masked tensor by its sum, like this:… Read More How can I normalize after masking?

# Is there a torch function to derive union of two tensors?

There is a function that can be used to derive union of two tensors in numpy, as below: import torch import numpy as np a = torch.tensor([0, 1, 2]) b = torch.tensor([2, 3, 4]) c = np.union1d(a, b) # c = array([0, 1, 2, 3, 4]) c = torch.from_numpy(c) # c = torch.tensor([0, 1, 2,… Read More Is there a torch function to derive union of two tensors?

# Pytorch, get the index of the first 0 in as mask?

I have a tensor that looks like: (1, 1, 1, 1, 1, 1, 1, 1, 0, 0). I want to get the index where the first zero appears. What would the be best way do this? >Solution : Not the best usage of argmin but it should work here I think: >>> torch.tensor([1, 1, 1,… Read More Pytorch, get the index of the first 0 in as mask?

# Is it possible to auto-size the subsequent input of a layer following torch.nn.Flatten within torch.nn.Sequential in PyTorch?

If I have the following model class for example: class MyTestModel(nn.Module): def __init__(self): super(MyTestModel, self).__init__() self.seq1 = nn.Sequential( nn.Conv2d(3, 6, 3), nn.MaxPool2d(2, 2), nn.Conv2d(6, 16, 3), nn.MaxPool2d(2, 2), nn.Flatten(), nn.Linear(myflattendinput(), 120), # how to automate this? nn.ReLU(), nn.Linear(120, 84), nn.ReLU(), nn.Linear(84, 2), ) self.softmax = nn.Softmax(dim=1) def forward(self, x): x = self.seq1(x) x = self.softmax(x)… Read More Is it possible to auto-size the subsequent input of a layer following torch.nn.Flatten within torch.nn.Sequential in PyTorch?

# Understanding broadcasting and arithmetic operations on different dimension tensors

I’m currently working on computing various similarity metrics between vectors such as cosine similarity, euclidean distance, mahalanobis distance, etc. As I’m working with vectors that can be very large, I need compute time to be minimal. I’m struggling to understand how to work with vectors of different dimensions (they, do, however, share one dimension) and… Read More Understanding broadcasting and arithmetic operations on different dimension tensors

# MSELoss from Pytorch

I am trying to train a neural network using Pytorch. I would like the loss function to be the MSE. I tried to use torch.nn.MSELoss, however I get an error that I do not understand. For example the following code gives me RuntimeError: Boolean value of Tensor with more than one value is ambiguous import… Read More MSELoss from Pytorch

# How to convert [a,b,c] to [a+b+c, b+c, c] in numpy?

If I have an array [a,b,c], how can I convert it to [a+b+c, b+c, c] in the most efficient way? (a, b, c) are floats. Thanks! >Solution : You can use np.cumsum: import numpy as np a = np.array([1, 2, 3]) print(np.cumsum(a[::-1])[::-1]) # Outputs [6 5 3]