Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed

import torch
from torch.autograd import Variable

x = Variable(torch.FloatTensor([11.2]), requires_grad=True)
y = 2 * x

print(x)
print(y)

print(x.data)
print(y.data)

print(x.grad_fn)
print(y.grad_fn)

y.backward() # Calculates the gradients

print(x.grad)
print(y.grad)

enter image description here
Error

C:\Users\donhu\AppData\Local\Temp\ipykernel_9572\106071707.py:2: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten\src\ATen/core/TensorBody.h:485.)
  print(y.grad)

Source code https://github.com/donhuvy/Deep-learning-with-PyTorch-video/blob/master/1.5.variables.ipynb

How to fix?

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

Call y.retain_grad() before calling y.backward().

The reason is because by default PyTorch only populate .grad for leaf variables (variables that aren’t results of operations), which is x in your example. To ensure .grad is also populated for non-leaf variables like y, you need to call their .retain_grad() method.

Also worth noting that it’s a warning rather than an error.

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading