Much if not all of what one needs to understand about confidence intervals can be learned through a simple analogy with a boy and his dog.
Imagine a boy and his dog outside together. The dog is friendly and there are no leash laws, so the boy lets his dog run free. The dog has been to obedience school and knows that he shouldn't stray too far: 68% of the time, he is within a certain distance of the boy, while he is within twice that distance 95% of the time.
While walking down the street, you meet the boy. You know the dog is somewhere nearby. Now, here comes the great insight! One day, you see the dog. You know the boy is somewhere nearby!
Think of the boy as a population mean and the dog as a sample mean. The standard error of the mean (SEM) plays the role of the "certain distance". The sample size reflects the dog's training. The larger the sample size, the smaller the "certain distance" and the better behaved the dog
|"Certain Distance"||Standard Error
of the Mean
Probability theory describes the way the dog wanders from the boy. That is, given a population mean (location of the boy), probability theory tells us how likely it is that a sample mean (location of the dog) will differ from the population mean by so many SEMs (wander from the boy).
Statistical theory turns everything around.
This is how confidence intervals work.