Suppose I have a tensor like [0.6, 0.7, 0.4] and a mask like: [1, 0,0] How can I normalize it to: [1,0,0] my try: normalized_attn_scores = F.softmax(attn_scores, 1) normalized_attn_scores = normalized_attn_scores.mul(attn_mask) But it doesn’t produce the desired output >Solution : You can normalize after masking by dividing the masked tensor by its sum, like this:… Read More How can I normalize after masking?
There is a function that can be used to derive union of two tensors in numpy, as below: import torch import numpy as np a = torch.tensor([0, 1, 2]) b = torch.tensor([2, 3, 4]) c = np.union1d(a, b) # c = array([0, 1, 2, 3, 4]) c = torch.from_numpy(c) # c = torch.tensor([0, 1, 2,… Read More Is there a torch function to derive union of two tensors?
I have a tensor that looks like: (1, 1, 1, 1, 1, 1, 1, 1, 0, 0). I want to get the index where the first zero appears. What would the be best way do this? >Solution : Not the best usage of argmin but it should work here I think: >>> torch.tensor([1, 1, 1,… Read More Pytorch, get the index of the first 0 in as mask?
I’m trying to understand the basic pytorch autograd system: x = torch.tensor(10., requires_grad=True) print(‘tensor:’,x) x.backward() print(‘gradient:’,x.grad) output: tensor: tensor(10., requires_grad=True) gradient: tensor(1.) since x is a scalar constant and no function is applied to it, I expected 0. as the gradient output. Why is the gradient 1. instead? >Solution : Whenever you are using value.backward(),… Read More Understanding gradient computation using backward() in PyTorch
If I have the following model class for example: class MyTestModel(nn.Module): def __init__(self): super(MyTestModel, self).__init__() self.seq1 = nn.Sequential( nn.Conv2d(3, 6, 3), nn.MaxPool2d(2, 2), nn.Conv2d(6, 16, 3), nn.MaxPool2d(2, 2), nn.Flatten(), nn.Linear(myflattendinput(), 120), # how to automate this? nn.ReLU(), nn.Linear(120, 84), nn.ReLU(), nn.Linear(84, 2), ) self.softmax = nn.Softmax(dim=1) def forward(self, x): x = self.seq1(x) x = self.softmax(x)… Read More Is it possible to auto-size the subsequent input of a layer following torch.nn.Flatten within torch.nn.Sequential in PyTorch?
I’m currently working on computing various similarity metrics between vectors such as cosine similarity, euclidean distance, mahalanobis distance, etc. As I’m working with vectors that can be very large, I need compute time to be minimal. I’m struggling to understand how to work with vectors of different dimensions (they, do, however, share one dimension) and… Read More Understanding broadcasting and arithmetic operations on different dimension tensors
I am trying to train a neural network using Pytorch. I would like the loss function to be the MSE. I tried to use torch.nn.MSELoss, however I get an error that I do not understand. For example the following code gives me RuntimeError: Boolean value of Tensor with more than one value is ambiguous import… Read More MSELoss from Pytorch
If I have an array [a,b,c], how can I convert it to [a+b+c, b+c, c] in the most efficient way? (a, b, c) are floats. Thanks! >Solution : You can use np.cumsum: import numpy as np a = np.array([1, 2, 3]) print(np.cumsum(a[::-1])[::-1]) # Outputs [6 5 3]