Fight, or hit with the fists / invite to fight
The connection between ‘dukes’ and ‘fists’ isn’t obvious at first glance, but it derives from Cockney rhyming slang: Duke of Yorks = forks = fingers/hands. In rhyming slang, the intended word is replaced by a phrase in which one word rhymes with it. A standard example is ‘china plate’ meaning ‘mate.’
‘Forks’ has been a slang term for ‘fingers’ or ‘hands’ since the 1700s. In Nathan Bailey’s Etymological English Dictionary, 1737, ‘fork’ is recorded as slang for ‘pickpocket.’
“FORK, a Pick-pocket. Lets Fork him; Let us pick that Man’s Pocket. It is done by thrusting the Fingers, strait, stiff, open and very quick into the Pocket, and so closing them, hook what can be held between them.”
This slang term makes use of the fact that fingers resemble the tines of a fork. The term ‘fork-out’, meaning ‘pay money’, comes from the same source and is recorded by 1831.
In 1859, The Vulgar Tongue: a Glossary of Slang, Cant, and Flash Words and Phrases attests the verb ‘fork’ in the phrase “fork out the tin” meaning “hand out the money,” used in London between 1839 and 1859. This puts ‘fork’ at the right place and time to make it possible for Cockney rhyming slang to invoke the Duke of York.
A citation in print of the expression ‘put up your dukes’ is in John C. Hotten’s Dictionary of Modern Slang, Cant, and Vulgar Words, 1874, and this also supports the ‘forks’ = ‘fingers’ notion:
“Dooks, or dukes, the hands, originally modification of the rhyming slang ‘Duke of Yorks,’ forks = fingers, hands… The word is in very common use among low folk.”
Saying “put up your dukes” these days is usually done as a joke. We still speak, though, of candidates “duking it out” in election years.
I suspect that whoever invented the forks we use based their shape on human fingers. Before that, we certainly used our fingers for the same purpose. And still do, when it comes to barbecued spareribs.