If we set the Boltzmann constant to $1$, then entropy would just be $\ln \Omega$, temperature would be measured in $\text{joules}$ ($\,\text{J}\,$), and average kinetic energy would be an integer times $\frac{T}{2}$. Why do we need separate units for temperature and energy?
One reason you might think $T$ should be measured in Joules is the idea that temperature is the average energy per degree of freedom in a system. However, this is only an approximation. That definition would correspond to something proportional to $\frac{U}{S}$ (internal energy over entropy) rather than $\frac{\partial U}{\partial S}$, which is the real definition. The approximation holds in cases where the number of degrees of freedom doesn't depend much on the amount of energy in the system, but for quantum systems, particularly at low temperatures, there can be quite a bit of dependence.

If you accept that $T$ is defined as $\frac{\partial U}{\partial S}$ then the question is about whether we should treat entropy as a dimensionless quantity. This is certainly possible, as you say.

But for me there's a very good practical reason not to do that: temperature is not an energy, in the sense that it doesn't, in general, make sense to add the temperature to the internal energy of a system or set them equal. Units are a useful tool for preventing you from accidentally trying to do such a thing.

In special relativity, for example, it makes sense to set $c=1$ because then it does make sense to set a distance equal to a time. By doing that, you're simply saying that the path between two points is light-like.

But $T=\frac{\partial U}{\partial S}$ measures the change in energy with respect to entropy. Entropy and energy are extensive quantities, whereas temperature is an intensive one. This means that it doesn't very often make sense to equate them without also including some non-constant factor relating to the system's size. For this reason, it's very useful to keep Boltzmann's constant around.

My personal favorite way to do it is to measure entropy in bits, so that $k_B = \frac{1}{\ln 2} \,\mathrm{bits}$ and the units of temperature are $\mathrm{J\cdot bits^{-1}}$. Having entropy rather than temperature as the quantity with the fundamental unit tends to make it much clearer what's going on, and bits are a pretty convenient unit in terms of building an intuition about the relationship to probability theory.
