Boron carbide (B4C) is an attractive material for numerous applications including vehicle armor, cutting tools, blasting nozzles, and abrasive powder, owing to its extreme hardness, high melting point, high Young’s modulus, and excellent thermoelectric properties. However, the application of B4C is limited by the high-temperature synthesis process. The present work aims to explore a low-temperature manufacturing process for synthesizing B4C with a small amount of free carbon. Poly(resorcinol borate) with an aromatic structure and high char yield was chosen as the aromatic polymeric precursor. A combination of Fourier transform infrared spectroscopy, X-ray photoelectron spectroscopy, X-ray diffraction, scanning electron microscopy, transmission electron microscopy and Raman spectroscopy was performed to investigate the influences of the reaction temperature and holding time on the changes in the precursor microstructure. The results indicate that the rod-like structure of crystalline B4C is successfully synthesized at 600 °C, and the free carbon can be reduced to about 0.8 wt% in the final product. This is because the pyrolysis temperature controlled the carbon content of the B4C, which led to an enlarged contact domain between B2O3 and carbon, and a relatively low-temperature synthesis of B4C.