I am new to pytorch and started with this github code. I do not understand the comment in line 60-61 in the code "because weights have requires_grad=True, but we don't need to track this in autograd". I understood that we mention requires_grad=True to the variables which we need to calculate the gradients for using autograd but what does it mean to be "tracked by autograd" ?
The wrapper with torch.no_grad() temporarily sets all of the requires_grad flags to false. An example is from the official PyTorch tutorial.
x = torch.randn(3, requires_grad=True)
print(x.requires_grad)
print((x ** 2).requires_grad)

with torch.no_grad():
    print((x ** 2).requires_grad)

Output:
True
True
False

I recommend you to read all the tutorials from the link above.
In your example: I guess the author does not want PyTorch to calculate the gradients of the new defined variables w1 and w2 since he just want to update their values.
