Q: Quantifying how informative knowledge of an observation's value on one variable for predicting its value on another Say I have two variables, V1 and V2:
df <- data.frame(V1 = c(1, 3, 3, 4, 2, 5, 6, 4, 8, 7),
                 V2 = c(4, 5, 4, 6, 1, 5, 6, 5, 5, 7))

These variables are positively correlated:
cor(df$V1, df$V2)
[1] 0.6385456

Meaning that knowledge of V1 alone is sufficient to predict V2 with reasonable accuracy, and vice versa. However, for some of these observations, knowledge of their value for V1 is very informative with respect to their value of V2 (and vice versa), whereas for others this is less the case. 
I would like to find some way to quantify for each observation the informativeness of knowing its value on one variable for predicting its value on the other variable and vice versa. Roughly, I'm thinking that this should be something like the difference between the predicted value of V1 given some specific value of V2 and the predicted value of V1 given V1's mean - i.e. something that captures what knowledge of this observation's V2 value adds to mere knowledge of V1's mean. 
But I am unsure if this is a good direction - any ideas on how to approach this?
