The equestrian sports in the Olympics are older than the Games themselves, even the ancient games. They were invented long before the first Olympics were ever held, in fact. All of these sports have their origins in the military- they were all invented and developed by the great armies of Europe and Asia. Dressage is the oldest of the 3 modern horse sports- its roots can be traced clear back to the time of Xenophon, who is considered by most historians to be the founder of modern horsemanship and equitation. Xenophon wrote a treatise on the subject of classical riding and horse training which contained principles that are still used today at places like the Spanish Riding School in Vienna, Austria. To give you an idea of how long ago this took place, Xenophon lived and died 400 years before the birth of Christ- and his treatise was published during the later years of his life, not after he died. He was Greek, and something of a jack of all trades during his lifetime- a philosopher, a soldier, a diplomat, and a statesman all rolled into one.
Through the centuries which followed Xenophon's life, dressage continued to evolve and develop- not as a sport, mind, but rather as a system of training horses and riders for mounted warfare. The famous leaps and jumps which the horses at the Spanish Riding School do, which are called the " airs above the ground", were all invented as movements for combat. As dressage continued to develop, what would eventually become the sport of 3 Day Eventing began to emerge alongside it. Eventing was also a product of the military- in fact, the original name for the sport was the Militare. This sport evolved out of and is descended from the tests which were given to cavalry officers and their mounts at or near the end of the officers' and mounts' training. In a very real sense, they were like a final exam- one which was meant to determine once and for all just who was fit to lead troops on the battlefield. Those who couldn't or didn't pass the tests (regardless of the reason why) were prohibited from leading troops in combat. The early events had 3 parts to them, just as today's modern events do- and all the horses and riders had to complete all 3 parts in in order to pass the final test.
Show jumping is a bit different. It too came from the military, as the other two sports did, but in this case it came as a result of the jumping contests which cavalry officers would get into. Almost every unit in most armies had a rivalry with one or more other units, and it was common for the officers in these groups to hold contests to see who could jump the highest or get around a course of fences in the least amount of time. The modern sport came about as a result of those early contests. ALL the horse sports were originally male only, and they were all invented by men. Women didn't start competing in the equestrian sports until the 1950's, and it took decades for them to gain an international competitive foothold. Once they did, however, they eventually came to dominate all the equestrian sports in the Olympics and competitions like them. Today, women outnumber men by a ratio of 7 to 1 in most of these sports. As for riding in the 18th and 19th centuries, it was similar to today, except that almost everyone learned to ride as they grew up back then. Horses were the main means of transport, and people learned to ride then the way that teenagers learn to drive cars in the modern age. Ditto for carriage driving- that too was a profession which people generally learned as they grew up.