  Many artificial intelligence models process input data of different lengths
and resolutions, making the shape of the tensors dynamic. The performance of
these models depends on the shape of the tensors, which makes it difficult to
optimize the tensors before the model runs. There are two common solutions to
this problem. The first is to add useless data to the input to match a
pre-optimized tensor library. The second is to use small basic tensors to
create a tensor that is closest in size to the input data and then tune it to
minimize padding. However, this second solution can be time-consuming.
  This paper proposes a new technique for deep learning compilers called
FTuner. Instead of using a large design space or training a cost model, we use
an abstract computational unit called the uKernel to patch together small,
various-sized tensors to match the shape of the input tensor. We determine the
shape of the uKernel using an analytic hardware information model. Experiments
show that the FTuner can achieve comparable operators and end-to-end
performance to vendor libraries and achieves 3\% speedup on existing auto-tuner
with the model-training compiler while reducing tuning time by two orders of
magnitude.
