  An effective action is proposed to compute the expectation value of Wilson
loops in $(S)U(N)$ gauge theories. The action consists of fermions localized on
the loop and an Abelian gauge field that fixes the representation. The
discussion is limited to weak coupling and Wilson loops in the fundamental
representation extended along a smooth curve, but there are no restrictions on
the matter content as long as the theory has a UV fixed point or it is
conformal. For a circular Wilson loop it is found that the expectation value
coincides at leading order with the exact result of the $1/2$ BPS Wilson loop
of ${\cal N}=4$ super Yang-Mills, which is determined by a solvable Gaussian
matrix model. This hints towards a universal connection to string theory duals
and SYK models.
