  This paper presents Decision Machines (DMs), an innovative evolution of
traditional binary decision trees, which leverages matrix computations to
significantly enhance both computational efficiency and interpretability. By
explicitly mapping the dependencies between predictions and binary tests within
a vector space, DMs offer a streamlined approach to navigating decision paths.
We integrate decision trees with kernel methods, ensemble methods and attention
mechanisms. The integration of these elements not only bolsters the
hierarchical structure of decision trees but also aligns with the computational
efficiency of matrix computations. Our work bridges the gap between traditional
machine learning algorithms and modern deep learning techniques, providing a
novel foundation for further research and application in the field of machine
learning.
