The easiest way to understand these types of questions is to imagine the extremes and interrogate those extremes. Let us imagine that we only have two data points in some x-y plane, and our goal is to estimate a linear regression model (a single point of data would tell us nothing so let’s ignore that case). If we want to estimate a linear regression model of two variables, we know we need to estimate an intercept and a slope. So if we have two data points, such a thing is possible! We actually have “perfectly fitted” our model, because r-squared will be 100%! However, we exhausted every bit of data we have, so we can’t really say anything about the data generating process that created our data. Maybe, if our initial line was a positively sloped thing, the real relationship between our variables is strongly negative. But we don’t know! We don’t have any data left to give us an idea. So if we got more data, we could adjust the intercept and slope parameters that we estimate to get a better idea. That’s effectively why degrees of freedom matter.
And if we add more parameters to estimate that takes from our “budget” of data we have. We might think, as we add parameters that we estimate, because our goodness of fit indicators (the basic ones, anyways) increase, that we’re getting closer to a good model! But r-squared can’t go down as you add parameters, and you’ve lost more data, yet again!
Degrees of freedom represent the number of independent values that are free to vary when estimating a statistical parameter.
Imagine a see-saw with 3 people sitting on it. You can put the first two people wherever you want, but the third person has to go in a very specific place to balance everything. Only two people have freedom, the 3rd person is constrained by your earlier choices of where you put the first two. Degrees of freedom are like the people who you can place freely. In this setup, even though you have 3 people, only 2 have freedom – 2 degrees of freedom. One person’s position must be locked in to maintain balance.
Comments
The easiest way to understand these types of questions is to imagine the extremes and interrogate those extremes. Let us imagine that we only have two data points in some x-y plane, and our goal is to estimate a linear regression model (a single point of data would tell us nothing so let’s ignore that case). If we want to estimate a linear regression model of two variables, we know we need to estimate an intercept and a slope. So if we have two data points, such a thing is possible! We actually have “perfectly fitted” our model, because r-squared will be 100%! However, we exhausted every bit of data we have, so we can’t really say anything about the data generating process that created our data. Maybe, if our initial line was a positively sloped thing, the real relationship between our variables is strongly negative. But we don’t know! We don’t have any data left to give us an idea. So if we got more data, we could adjust the intercept and slope parameters that we estimate to get a better idea. That’s effectively why degrees of freedom matter.
And if we add more parameters to estimate that takes from our “budget” of data we have. We might think, as we add parameters that we estimate, because our goodness of fit indicators (the basic ones, anyways) increase, that we’re getting closer to a good model! But r-squared can’t go down as you add parameters, and you’ve lost more data, yet again!
If a building is sitting on the ground no matter what you do , it can’t go down.
What its free to do are things like fly away, fall over or go sideways. Those are it’s degrees of freedom.
If we add different kinds of supports we can remove one or more of the degrees of freedom until everything is fixed.
Degrees of freedom represent the number of independent values that are free to vary when estimating a statistical parameter.
Imagine a see-saw with 3 people sitting on it. You can put the first two people wherever you want, but the third person has to go in a very specific place to balance everything. Only two people have freedom, the 3rd person is constrained by your earlier choices of where you put the first two. Degrees of freedom are like the people who you can place freely. In this setup, even though you have 3 people, only 2 have freedom – 2 degrees of freedom. One person’s position must be locked in to maintain balance.