  Live video broadcasting normally requires a multitude of skills and expertise
with domain knowledge to enable multi-camera productions. As the number of
cameras keep increasing, directing a live sports broadcast has now become more
complicated and challenging than ever before. The broadcast directors need to
be much more concentrated, responsive, and knowledgeable, during the
production. To relieve the directors from their intensive efforts, we develop
an innovative automated sports broadcast directing system, called Smart
Director, which aims at mimicking the typical human-in-the-loop broadcasting
process to automatically create near-professional broadcasting programs in
real-time by using a set of advanced multi-view video analysis algorithms.
Inspired by the so-called "three-event" construction of sports broadcast, we
build our system with an event-driven pipeline consisting of three consecutive
novel components: 1) the Multi-view Event Localization to detect events by
modeling multi-view correlations, 2) the Multi-view Highlight Detection to rank
camera views by the visual importance for view selection, 3) the
Auto-Broadcasting Scheduler to control the production of broadcasting videos.
To our best knowledge, our system is the first end-to-end automated directing
system for multi-camera sports broadcasting, completely driven by the semantic
understanding of sports events. It is also the first system to solve the novel
problem of multi-view joint event detection by cross-view relation modeling. We
conduct both objective and subjective evaluations on a real-world multi-camera
soccer dataset, which demonstrate the quality of our auto-generated videos is
comparable to that of the human-directed. Thanks to its faster response, our
system is able to capture more fast-passing and short-duration events which are
usually missed by human directors.
