  The main goal of this paper is to develop an estimate for the entropy of
random stationary ergodic symbolic sequences with elements belonging to a
finite alphabet. We present here the detailed analytical study of the entropy
for the high-order Markov chain in the bilinear approximation. The appendix
contains a short comprehensive introduction into the subject of study.
