I've used Hyperopt in Python, but I'm looking for a package with similar capabilities in R. Does a package like this exist?
GRUs and LSTMs utilize different approaches toward gating information to prevent the vanishing gradient problem. Here are the main points comparing the two:

The GRU unit controls the flow of information like the LSTM unit, but without having to use a memory unit. It just exposes the full hidden content without any control.
GRUs are relatively new, and in my experience, their performance is on par with LSTMs,  but computationally more efficient (as pointed out, they have a less complex structure). For that reason, we are seeing it being used more and more.

For a detailed description, you can explore this research paper on Arxiv. The paper explains all this brilliantly.
You can also explore these blogs for a better idea:

WildML
Colah - Github

Hope that helps!
