The Oregon National Caves Monument is one of the only marble caves in the world. The caves are estimated to be 1-2 million years old. They were discovered in the late 1800's and have seen their share of visitors since. The closest town is Cave Junction. Even though it's a mere 19 miles, it takes over 40 minutes to drive there on the windy road. In order to view the caves, you need to purchase a ticket for a guided tour. I predict that our ranger tour guide was a newbie.... or a robot. He was very rehearsed and his jokes seemed automatic without any character or inflection behind them. He was nice. So we walked through the caves in a group of 20 or so. There are lighting fixtures in place so that you can see the different formations. The tour guide will explain to you how to identify different cave structures. The tour is about 45 minutes long and filled with fun facts.
After the hike, walk around the surrounding trails a bit. There a great view over the Siskiyou Mountains. Also, check out the Chateau. It was built in 1934 and is still a beautiful building, and still a lodging option to this day!
I recommend hitting this spot up if you're on your way to the Redwoods. That was how we did it. Otherwise, it's not really close to anything.