  Edge computing has been emerging as a popular scenario for model inference.
However, the inference performance on edge devices (e.g., Multi-Core DSP, FGPA,
etc.) suffers from inefficiency due to the lack of highly optimized inference
frameworks. Previous model inference frameworks are mainly developed in an
operator-centric way, which provides insufficient acceleration to edge-based
inference. Besides, the operator-centric framework incurs significant costs for
continuous development and maintenance.
  In this paper, we propose Xenos, which can automatically conduct
dataflow-centric optimization of the computation graph and accelerate inference
in two dimensions. Vertically, Xenos develops operator linking technique to
improve data locality by restructuring the inter-operator dataflow.
Horizontally, Xenos develops DSP-aware operator split technique to enable
higher parallelism across multiple DSP units. Our evaluation proves the
effectiveness of vertical and horizontal dataflow optimization, which reduce
the inference time by 21.2\%--84.9\% and 17.9\%--96.2\% , respectively.
Besides, Xenos also outperforms the widely-used TVM by
3.22$\times$--17.92$\times$. Moreover, we extend Xenos to a distributed
solution, which we call d-Xenos. d-Xenos employs multiple edge devices to
jointly conduct the inference task and achieves a speedup of 3.68x--3.78x
compared with the single device.
