Covariance

Covariance answers a directional question: when one random variable is above its mean, does the other also tend to be above its mean? Positive covariance suggests they move together on average. Negative covariance suggests offsetting movement.

That makes covariance a dependence summary, not a full description of the joint distribution. Two variables can have the same covariance and still behave very differently overall.

Using the same joint table as the joint-distributions page, E[X]=0.50, E[Y]=0.70, and E[XY]=0.40 because XY is 1 only at the outcome (1,1). That gives Cov(X,Y)=0.40-0.50(0.70)=0.05.

The number is small but positive, which matches the table: the outcome where both variables equal 1 has relatively high probability. If you standardize by the two standard deviations, you get a positive correlation as well, but the main Exam P point is recognizing how covariance changes the behavior of sums.

Covariance is one of the main gates into multivariate calculation. It appears directly in its own right and indirectly whenever a problem asks for the variance of X+Y, X-Y, or another linear combination.

That is why this topic is higher value than it first looks. Even when the final question is about a variance or probability, the real blocker is often whether you noticed that dependence adds or subtracts a covariance term.

The most common mistake is dropping the covariance term from the variance of a sum. Another is assuming that covariance zero always means independence. It often does in simple examples, but it is not a general equivalence.

Covariance is the raw ingredient behind covariance matrices, principal directions, portfolio variance, and many multivariate models. Exam P covers the two-variable version, but the intuition scales upward into much of statistics and ML.