Reddit lallapalalable: Early man tracked time day by day using apparent time, or time based on sundials and other observations of nature. I can't find too much info right now but as far as I can tell there was no standard unit of time shorter than a day among any major civilization for a while.
The Egyptians defined hours as 1/12th of either day or night, and had seasonal variations on the length of their hour.
Greek astronomers were the first to establish the modern hour, by dividing the day into six parts and then dividing those parts into four more. They also had an early version of the minute, which was how long it took for the sun to travel one degree along the sky, or about four minutes.
The Babylonians went a little nuts, also dividing the day into six parts, but then divided each part by sixty for their sub-units, up to at least six subdivisions, the smallest individual units being as accurate as two microseconds. However, instead of using a 1/24th of a day hour like the Greeks, they had a 1/12th day hour (120 min), but did use the 1/360th day minute, and something resembling a second called the barelycorn, about 3.5 modern seconds and still used in the hebrew calendar today as the helek.
In 1000, a Persian scholar named al-Biruni first termed the word second when he defined the period of time between two new moons as a figure of days, hours, minutes, seconds, thirds, and fourths. The minute was the first subdivision of the hour by 60, then the second, and so on. Roger Bacon did this again in the 1200's, but started with hours, giving a more accurate figure. The term third still exists in some languages, such as Polish, but fourths were apparently too small for any practical use and fell out of style with the general population.
The late 1500s what the first time a true standard second came to being with the advent of mechanical clocks, so that the time could be measured objectively from mean-time instead of deriving it from the apparent-time. The first clock with a second hand was built between 1560-70and 1579 saw the first clock with actual markings denoting the seconds. However, they weren't very accurate, and the second remained arbitrary from machine to machine and unable to be reliably measured.
In 1644 it was realized that a pendulum of a specific length would have an oscillation period of exactly two seconds, and by 1670 William Clement had tinkered with the physics enough to build a clock precise enough that the second was now an established unit of time. By 1862 it was established that the second would be the base unit of time for all scientific research, along with the millimeter and milligram, by The British Association for the Advancement of Science (BAAS), defining the second as 1/86,400th of a solar day by the 1940s.
From there we've just been refining the accuracy of what we call a second, accounting for the Earth's axis and ever-so slowly declining rotational velocity, to the point where it's not even measured by observations based on the earth and sun, but by the distance light travels in a vacuum or how many vibrations a cesium atom makes. We've divided time so hard that to do so any further wouldn't make sense; events in subatomic physics just don't happen quickly enough for smaller units of time to be measured.
But yeah, that's where we got our units of time. It's such a ubiquitous thing that we've had literally our entire existence to hash it all out, and while today we're all in agreement about the standard subdivisions and have been for a long time, there were discrepancies in the past and in the context of your post it must have been a special kind of frustrating trying to figure out what two people mean by "hour"
*However, while we all agree on the day/hour/second situation, the annual calendar is still pretty sporadic in it's appearance. The lunar month may seem standard, but there are many South-East Asian cultures that have their own unique way of dividing the year. I won't get into that, because it's mostly a headache and I'm kinda losing steam on this anyway, but long story short we're far from having a universal system here.