The path-integral analysis of an associative memory model storing an infinite number of finite limit cycles
Kazushi Mimura
It is shown that an exact solution of the transient dynamics of an associative memory model storing an infinite number of limit cycles with $l$ finite steps by means of the path-integral analysis. %�L���X�e�b�v�n��z�N���f���̑z�N�ߒ����o�H�ϕ��ɂ���Č����ɉ�͂���D % Assuming the Maxwell construction ansatz, we have succeeded in deriving the stationary state equations of the order parameters from the macroscopic recursive equations with respect to the finite-step sequence processing model which has retarded self-interactions. %---modified---We derived the stationary state equations given by the path-integral analysis. %�z�N����̓������������Ԃ��L�q���鋐����ԕ�������z�N�ߒ����瓱�o����D % We have also derived the stationary state equations by means of the signal-to-noise analysis (SCSNA). %---ADDED--- The signal-to-noise analysis must assume that crosstalk noise of an input to spins obeys a Gaussian distribution. On the other hand, the path-integral method does not require such a Gaussian approximation of crosstalk noise. We have found that both the signal-to-noise analysis and the path-integral analysis give the completely same result with respect to the stationary state in the case where the dynamics is deterministic, when we assume the Maxwell construction ansatz. %---ADDED--- %���̒���Ԃ��L�q���鋐����ԕ������́C�ȑO��X�����߂�S/N��͂ɂ���͌��ʂƈ�v�����D %���̂悤�ɁCS/N��͂ł̓N���X�g�[�N�m�C�Y���K�E�X���z�ɏ]���Ɖ��肷��ɂ�������炸�C %����Ԃɂ����Ă̓K�E�X�ߎ����s�K�v�Ȍo�H�ϕ��ɂ���͌��ʂƈ�v�����D % %---DELETED---This means that the result of the signal-to-noise analysis (SCSNA) is an exact solution in the case where $T=0$. %�܂�CS/N��͂ɂ�����Ԃ̉�͌��ʂ��������ł��邱�Ƃ��킩�����D % \par We have shown the dependence of storage capacity ($\alpha_c$) on the number of patterns per one limit cycle ($l$). %�܂��C����Ԃ��L�q���鋐����ԕ���������C�n��̃X�e�b�v���ƋL���e�ʂ̊֌W�����߂��D % At $l=1$, storage capacity is $\alpha_c=0.138$ like the Hopfield model's. %�X�e�b�v����$l=1$�̂Ƃ��́C�L���e�ʂ�Hopfield���f���Ɠ���$\alpha_c=0.138$�ƂȂ�D % Storage capacity monotonously increases with the number of steps, and converges to $\alpha_c=0.269$ at $l\simeq 10$. %�X�e�b�v���̑����ƂƂ��ɒP���ɋL���e�ʂ͑������C %�X�e�b�v��$l=10$���x�ŁC�����ɖ����X�e�b�v�n��z�N���f���̋L���e��$\alpha_c=0.269$�Ɏ�������D % The original properties of the finite-step sequence processing model appear as long as the number of steps of the limit cycle has order $l=O(1)$. %�X�e�b�v��$l=O(1)$�ł̂ݗL���X�e�b�v�ɓ��L�̐����������Ƃ��킩�����D
JPhysA03.tex

\documentclass[preprint]{revtex4}

\usepackage{graphicx}
\newcommand{\sgn}{\,{\rm sgn}\,}
\newcommand{\mod}{\,{\rm mod}\,}
\newcommand{\Tr}[1]{\,\mathop{\rm Tr}_{#1}\,}
\renewcommand{\vec}[1]{\mbox{\boldmath $#1$}}
%\renewcommand{\baselinestretch}{1.0}


% Advenced Mean Field Methods, edited by Opper Saad. 
% T. Plefka, convergence condition ..., J Phys A 15, 1971 (1982).


\begin{document}

\title{The path-integral analysis of an associative memory model storing an infinite number of finite limit cycles}
\author{Kazushi Mimura}
\affiliation{
Faculty of Information Sciences, Hiroshima City University, 
Ohtsukahigashi 3-4-1, Asaminami-ku, Hiroshima, 731-3194 Japan, \\
Kobe City College of Technology, 
Gakuenhigashi-machi 8-3, Nishi-ku, Kobe, Hyogo, 651-2194 Japan}
\email{mimura@cs.hiroshima-cu.ac.jp}
\author{Masaki Kawamura}
\affiliation{Faculty of Science, Yamaguchi University, Yoshida 1677-1,
Yamaguchi, 753-8512 Japan}
\author{Masato Okada}
\affiliation{
Laboratory for Mathematical Neuroscience,
RIKEN Brain Science Institute, Saitama 351-0198, Japan, \\
"Intelligent Cooperation and Control", PRESTO, JST, 
c/o RIKEN BSI, Saitama 351-0198, Japan, 
%ERATO Kawato Dynamic Brain Project, 
%2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan 
}

\date{\today}

\begin{abstract}
It is shown that an exact solution of the transient dynamics of an associative memory model 
storing an infinite number of limit cycles with $l$ finite steps by means of the path-integral analysis. 
%�L���X�e�b�v�n��z�N���f���̑z�N�ߒ����o�H�ϕ��ɂ���Č����ɉ�͂���D
%
Assuming the Maxwell construction ansatz, we have succeeded in deriving the stationary state equations of the order parameters 
from the macroscopic recursive equations with respect to the finite-step sequence processing model which has retarded self-interactions. 
%---modified---We derived the stationary state equations given by the path-integral analysis. 
%�z�N����̓������������Ԃ��L�q���鋐����ԕ�������z�N�ߒ����瓱�o����D
%
We have also derived the stationary state equations by means of the signal-to-noise analysis (SCSNA). %---ADDED---
The signal-to-noise analysis must assume that crosstalk noise of an input to spins obeys a Gaussian distribution. 
On the other hand, the path-integral method does not require such a Gaussian approximation of crosstalk noise. 
We have found that both the signal-to-noise analysis and the path-integral analysis 
give the completely same result with respect to the stationary state in the case where the dynamics is deterministic, when we assume the Maxwell construction ansatz. %---ADDED---
%���̒���Ԃ��L�q���鋐����ԕ������́C�ȑO��X�����߂�S/N��͂ɂ���͌��ʂƈ�v�����D
%���̂悤�ɁCS/N��͂ł̓N���X�g�[�N�m�C�Y���K�E�X���z�ɏ]���Ɖ��肷��ɂ�������炸�C
%����Ԃɂ����Ă̓K�E�X�ߎ����s�K�v�Ȍo�H�ϕ��ɂ���͌��ʂƈ�v�����D
%
%---DELETED---This means that the result of the signal-to-noise analysis (SCSNA) is an exact solution in the case where $T=0$. 
%�܂�CS/N��͂ɂ�����Ԃ̉�͌��ʂ��������ł��邱�Ƃ��킩�����D
%
\par
We have shown the dependence of storage capacity ($\alpha_c$) on the number of patterns per one limit cycle ($l$). 
%�܂��C����Ԃ��L�q���鋐����ԕ���������C�n��̃X�e�b�v���ƋL���e�ʂ̊֌W�����߂��D
%
At $l=1$, storage capacity is $\alpha_c=0.138$ like the Hopfield model's. 
%�X�e�b�v����$l=1$�̂Ƃ��́C�L���e�ʂ�Hopfield���f���Ɠ���$\alpha_c=0.138$�ƂȂ�D
%
Storage capacity monotonously increases with the number of steps, and converges to $\alpha_c=0.269$ at $l\simeq 10$. 
%�X�e�b�v���̑����ƂƂ��ɒP���ɋL���e�ʂ͑������C
%�X�e�b�v��$l=10$���x�ŁC�����ɖ����X�e�b�v�n��z�N���f���̋L���e��$\alpha_c=0.269$�Ɏ�������D
%
The original properties of the finite-step sequence processing model appear as long as the number of steps of the limit cycle has order $l=O(1)$. 
%�X�e�b�v��$l=O(1)$�ł̂ݗL���X�e�b�v�ɓ��L�̐����������Ƃ��킩�����D
\end{abstract}

\keywords{Associative Memory Model, Limit Cycles, Path-Integral Method, Stationary State, Exact Solution}
\maketitle








\section{Introduction}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

%�����n��z�N���f���@infinite-step sequence processing model
%�L���n��z�N���f���@  finite-step sequence processing model

In recent years, theories that can analyze the transient dynamics have been discussed for systems with frustrations especially a correlation-type associative memory 
\cite{hopfield1982,shiino1992,amit1985b,kuhn1991,mimura1995,okada1996,mimura1996,mimura1998a,mimura1998b,mimura2003,okada1995}. 
%�ߔN�C�t���X�g���[�V���������n�C���ɑ��֌^�A�z�L�����f���Ɋւ��ē��I���_�̋c�_������ɍs���Ă���\cite{hopfield1982}-\cite{okada1995}�D
%
D${\rm \ddot{u}}$ring et al presented a path-integral method for an infinite-step sequence processing model and analyzed the properties of the stationary state \cite{during1998}. 
%During�͖����X�e�b�v�n��z�N���f���Ɍo�H�ϕ���K�p���C���̒���ԕ���������L���e�ʂȂǂ����߂�\cite{during1998}�D
%
By using D${\rm \ddot{u}}$ring et al's analysis, Kawamura and Okada succeeded in deriving an exact macroscopic description of the transient dynamics \cite{kawamura2002}. 
%Kawamura��Okada��During�̒莮���Ɋ�Â��ߓn�������L�q���鋐����ԕ����������߂�\cite{kawamura2002}�D
%
The transient dynamics can be analyzed not only by using the path-integral method \cite{coolen1993,coolen2000} but by using the signal-to-noise analysis, e.g., statistical neurodynamics \cite{amari1988a,amari1988b}�D
%�A�z�L�����f���̓������́C�o�H�ϕ�\cite{coolen1993,coolen2000}�����łȂ����v�_�o�͊w�ƌĂ΂��S/N��͂ɂ���Ă���͂ł���\cite{amari1988a,amari1988b}�D
%
The signal-to-noise analysis is an approximation theory in which crosstalk noise obeys a Gaussian distribution. 
%���v�_�o�͊w�́C�N���X�g�[�N�m�C�Y���K�E�X���z�ŋߎ����ċ�����Ԃ�\���Q�����𓱂��ߎ����_�ł���D
%
On the other hand, the path-integral method does not require such a Gaussian approximation of crosstalk noise. 
%����C�o�H�ϕ��ɂ���͂ł́C�N���X�g�[�N�m�C�Y�̃K�E�X�ߎ��͕s�K�v�ł���D
%
However surprisingly, the macroscopic equations of the exact solution given by means of the path-integral method are completely equivalent to those of the signal-to-noise analysis with respect to this model. 
%�ɂ�������炸�C�����ׂ����Ƃɖ����X�e�b�v�n��z�N���f���ɂ��ẮC
%�������ł���o�H�ϕ��ɂ�錋�ʂƁC�ߎ����_�ł���S/N��͂̌��ʂ͊��S�Ɉ�v����D
%
\par
It has turned out that the infinite-step sequence processing model can be more easily analyzed than the Hopfield model even if it is necessary to treat the dynamical process directly. 
%�����X�e�b�v�n��z�N���f���ł́C���I�ȉߒ��𒼐ڎ戵���K�v������ɂ��ւ�炸�C���ȑz�N���f���������_�I�Ȏ戵�����e�Ղł���D
%
This reason is as follows. 
The retrieval state of the infinite-step sequence processing model has no equilibrium state. 
Therefore, the correlations of the system are not very complex. 
%����́C�n��z�N���f���ł͓����L���p�^�[������H�̏�ԂƂ���2�x�ȏ���Ȃ����߂ɁC�n�̓��v�I�Ȑ������ȒP�ɂȂ邽�߂ł���D
%
Since the Hopfield model takes the same states repeatedly, 
its statistical properties are more complex than the infinite-step sequence processing model. 
%�������C���ȑz�N���f���ł́C�n��������Ԃ����x���Ƃ邽�߂ɁC���ԓI�ȑ��ւ����G�ɂȂ�D
%
Gardner et al analyzed the transient dynamics of the Hopfield model by using the path-integral method in the case where the dynamics is deterministic \cite{gardner1987}. 
%Gardner��́C�o�H�ϕ��I�ȕ��@�ɂ��C���ȑz�N���f���̃_�C�i�~�N�X����͂����D
%
They obtained the macroscopic equations of the transient dynamics at time step $t$ using $O(t^2)$ macroscopic variables and also obtained the macroscopic equations of the equilibrium state from the transient dynamics. 
%����_�I�_�C�i�~�N�X�̏ꍇ�ɂ��āCGardner��̉�͂́C�C�ӂ�$t$�X�e�b�v�ł�$O(t^2)$�̋����I�ϐ����g���ċ����I��ԕ������𓾂��D
%
These are equivalent to replica symmetric (RS) solutions given by using the replica method \cite{gardner1987}. 
%�K�[�h�i�[��̌��ʂ́C���v���J�@��RS���Ɉ�v����\cite{gardner1987}�D
%
%---DELETED--- On the other hand, it is well known that the RS solutions are equivalent to those of the signal-to-noise analysis \cite{shiino1992}. 
%---DELETED--- %����CRS����s/n��͂ɂ����ƈ�v���邱�Ƃ��悭�m���Ă���D
%---DELETED--- %
%---DELETED--- However until now, the relation between the path-integral method and the signal-to-noise analysis was unknown, 
%---DELETED--- because of the complexity of the statistical property. 
%---DELETED--- %����܂ŁC���v�I���������G�Ŏ�舵���̓���i���ȑz�N���f���ɂ��Ắj�C�o�H�ϕ���S/N��͂̊֌W�͕s���ł������D
%---DELETED--- %
Recently in the Hopfield model, Bolle et al compared the transient dynamics of the path-integral method with those of the signal-to-noise analysis only for a few time steps in the dynamics \cite{bolle2003}. 
%�����ŋ߁CBolle��́C���ȑz�N���f����S/N��͂ƌo�H�ϕ��ŉ�͂��Č��ʂ��r����\cite{bolle2003}�D
%
They have pointed out that the signal-to-noise analysis is exact up to time step 3 and inexact to step 4 or beyond. 
%�ނ�́C�z�N�ߒ���3�X�e�b�v�ڂ܂ł͗��҂̌��ʂ͈�v���C4�X�e�b�v�ڈȍ~�͂��ꂪ������Ƃ̌��ʂ�񍐂��Ă���D
%
\par
In order to discuss the relation between the path-integral method and the signal-to-noise analysis in more detail, 
we analyze a finite-step sequence processing model, which includes the Hopfield model and the infinite-step sequence processing model in special cases. 
%�����Ŗ{�_���ł́C�����X�e�b�v�n��z�N���f���Ǝ��ȑz�N���f�������ȏꍇ�Ƃ��Ċ܂ށC
%�L���X�e�b�v�n��z�N���f�����o�H�ϕ�����ĉ�͂��C���ȑz�N���f���ł̌o�H�ϕ���S/N��͂̊֌W���c�_����D
%
In the finite-step sequence processing model, the steady states of the system become limit cycles. 
%�L���X�e�b�v�n��z�N���f���ł́C�n�̒���Ԃ̓��~�b�g�T�C�N���ƂȂ�D
%
Since the finite-step sequence processing model can store limit cycles in the dynamics, 
the properties of the system are periodic and dynamic essentially like the infinite-step sequence processing model. 
%�L���X�e�b�v�n��z�N���f���̓��~�b�g�T�C�N�����L�������Ă��邽�߂ɁC
%�����X�e�b�v�n��z�N���f���Ɠ��l�Ɍn�̐����͖{���I�ɓ��I�ł���D
%
Moreover, the statistical properties of the finite-step sequence processing model are more complex than the infinite-step one. 
Since the period of the limit cycle is finite, the network takes the same states repeatedly. 
%�܂����~�b�g�T�C�N���̎������L���ł��邽�߂ɁC��H�̓�����Ԃ����x����邽�߁C���ȑz�N���f���Ɠ��l�̓��v�I���ւ�������D
%
Namely, the finite-step sequence processing model has the theoretical difficulties of both the Hopfield model and the infinite-step sequence processing model. 
%�܂�C�L���X�e�b�v�n��z�N���f���́C���ȑz�N���f���ƌn��z�N���f���̗��_�I�ȓ���𗼕������Ă���D
%
In this point of view, it would be very interesting to theoretically discuss the properties of the finite-step sequence processing model. 
%���̊ϓ_�ɂ����Ă��C���̃��f���̐����𗝘_�I�ɋc�_���邱�Ƃ͑�ϋ����[���D
%
\par
In this paper, we have exactly derived the transient dynamics of macroscopic recursive equations 
with respect to the finite-step sequence processing model by means of the path-integral analysis. %---ADDED---
%�o�H�ϕ��@�ɂ���ē����錋�ʂ́C������Ԃ�transient dynamics �ł���D
%
Until now, only in the infinite-step sequence processing model, which has no self-interactions, 
D${\rm \ddot{u}}$ring et al derived the stationary state equations of the order parameters by using the path-integral analysis \cite{during1998}. %---ADDED---
%����܂ŁCextensive loading �̏ꍇ($\alpha \ne 0$)�ɂ��ẮCtransient dynamics ����C
%����ԕ������𓱏o�� infinite-step sequence processing model �ɂ����Ă����������Ă��Ȃ� \cite{during1998}�D
%
The transient dynamics of various disordered systems can be also analyzed by using the path-integral method. 
%In the recent research, the path-integral analysis is adopted to various problems, e.g., minority game \cite{coolen2001}, on-line learning \cite{inoue2001} and so on. 
%�o�H�ϕ��@�́C���Ɉ�ʓI�ȉ�͕��@�ł���C�ŋ߂ł̓}�C�m���e�B�Q�[��\cite{coolen2001}��online�w�K���_\bibitem{inoue2001}�Ȃǂւ̓K�p�����łɂȂ���Ă���D
%
Therefore, it is important to derive stationary state equations of the order parameters from the macroscopic recursive equations. 
%���̊ϓ_������CMaxwell construction �𗘗p����Ƃ͂������̂́Ctransient dynamics ����̒���Ԃ̓��o�͏d�v�ł���D
%
Assuming the Maxwell construction ansatz, we have succeeded in deriving the stationary state equations 
from the macroscopic recursive equations with respect to the model, which has self-interactions, i.e., the finite-step sequence processing model. %---ADDED---
%��X�� retarded self-interaction ������ꍇ�ɂ��Ă�����ԕ������𓱂����D
%
%---DELETED---In the case where the dynamics is deterministic, we also derived the macroscopic equations for stationary states. 
\par
We also analyzed the finite-step sequence processing model by means of the signal-to-noise analysis (SCSNA). %\cite{mimura1995}
The stationary state equations given by the path-integral analysis are equivalent to those of the signal-to-noise analysis. 
%�L���X�e�b�v�n��z�N���f�����o�H�ϕ��ɂ���ĉ�͂��ē���ꂽ���ʂ́C
%$t\to\infty$,$T\to0$�ł́C�ȑO��X�����߂�S/N��͂ɂ���͌��ʂƈ�v����D
%
%---DELETED---It is very interesting that the path-integral method, which does not require a Gaussian approximation of crosstalk noise, 
%---DELETED---and the signal-to-noise analysis, which needs such an approximation, give the same results in the stationary state. 
%S/N��͂ł́C�N���X�g�[�N�m�C�Y�̃K�E�X�ߎ����K�v�ł���ɂ��ւ�炸�C
%����Ԃɂ����Ă̓K�E�X�ߎ����s�K�v�Ȍo�H�ϕ��ɂ���͌��ʂƈ�v���邱�Ƃ͂ƂĂ������[���D
%
This result corresponds to the fact that the replica method and the signal-to-noise analysis give completely equivalent results in the stationary state analysis of the Hopfield model. 
%����́C����܂ł̎��ȑz�N���f���̒���Ԃ̉�͂ŁC
%���v���J�@�Ȃǂɂ���͌��ʂƁCS/N��͂ɂ���͌��ʂ���v���邱�ƂɑΉ����Ă���D
%
Namely, the transient dynamics given by the signal-to-noise analysis gives an exact solution in both the stationary state and the first few time steps in the dynamics. 
%�܂�C���v�_�o�͊w�Ȃǂ�S/N��͂ɂ�铮�����̉�͂ɂ��C
%�����̐��X�e�b�v�ƒ���ԂɊւ��Ă͌o�H�ϕ��Ɠ��l�Ɍ����ɉ�͂ł��邱�Ƃ��킩�����D
%
%---DELETED--- This includes Gardner et al's result concerning the Hopfield model, i.e., 
%---DELETED--- the property of the equilibrium state given by a similar way to the path-integral method 
%---DELETED--- is equivalent to those of the signal-to-noise analysis \cite{gardner1987}. 
%---DELETED--- %���̌��ʂ́CGardner�̌��ʂ��܂�ł���D
%---DELETED--- %









\section{Definitions}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Let us consider a system storing an infinite number of limit cycles with $l$ finite steps.
The system consists of $N$ Ising-type spins (or neurons) $\sigma_i=\pm 1$. 
We consider the case where $N \to \infty$. 
The spins updates the state synchronously with the probability: 
\begin{equation}%(dynamics)
{\rm Prob}[\sigma_i(t+1)=-\sigma_i(t)]=\frac 12 \biggl[ 1 - \sigma_i(t) \tanh \beta h_i(t) \biggr] ,
\label{eq:dynamics}
\end{equation}
\begin{equation}%(internal potential)
h_i(t)=\sum_{j=1}^N J_{ij} \sigma_j(t) + \theta_i(t) ,
\end{equation}
where $\beta$ is the inverse temperature, $\beta = 1/T$. 
When the temperature is $T=0$, the updating rule of the state is deterministic. 
The term $\theta_i(t)$ is a time-dependent external field which is introduced in order to define a response function. 
The interaction $J_{ij}$ stores $p$ random patterns $\vec{\xi}^{\nu,\mu}=(\xi_1^{\nu,\mu},\cdots,\xi_N^{\nu,\mu})^T$ so as to retrieve the patterns as: 
\begin{equation}
\vec{\xi}^{\nu,1} \to \vec{\xi}^{\nu,2} \to \cdots \to \vec{\xi}^{\nu,l} \to \vec{\xi}^{\nu,1}, \nonumber
\end{equation}
sequentially for any $\mu$th limit cycle. 
For instance, the entries of the interaction matrix $\vec{J}=(J_{ij})$ are given by 
\begin{equation}%(the entries of the interaction matrix)
J_{ij}=\frac 1N \sum_{\nu =1}^{p/l} \sum_{\mu =1}^l \xi_i^{\nu ,\mu +1} \xi_j^{\nu ,\mu},
\end{equation}
where the pattern index $\mu$ are understood to be taken modulo $l$. 
Since the number of limit cycles is $p/l$, the total number of stored patterns is $p$. 
The number of stored patterns $p$ is given by $p=\alpha N$, where $\alpha$ is called the loading rate. 
In our analysis, the number of steps for each limit cycle $l$ is kept finite. 
Each component of the patterns is assumed to be an independent random variable that takes a value of either $+1$ or $-1$ according to the probability: 
\begin{equation}%(each component of the patterns)
{\rm Prob}[\xi_i^{\nu,\mu}=\pm 1]=\frac 12.
\end{equation}
%We determine the initial state $\vec{\sigma}(0)$ according to the following probability distribution 
%\begin{equation}%(initial state)
%{\rm Prob}[\sigma_i(0)=\pm 1]=\frac {1\pm m(0)\xi_i^{1,l}}2.
%\end{equation}
%Therefore, the overlap between the pattern $\vec{\xi}^{1,l}$ and the initial state $\vec{\sigma}(0)$ is $m(0)$. 
%The matrix $\vec{J}$ is not non-symmetric. 
For the subsequent analysis, the matrix $\vec{J}$ is represented as 
\begin{equation}%(interaction matrix)
\vec{J} = \frac 1N \vec{\xi}^T \vec{S} \vec{\xi},
\end{equation}
where the $p \times N$ matrix $\vec{\xi}$ is defined as 
\begin{equation}
\vec{\xi}=(\vec{\xi}^{1,1} \cdots \vec{\xi}^{1,l} \vec{\xi}^{2,1} \cdots \vec{\xi}^{2,l} \cdots \vec{\xi}^{p/l,1} \cdots \vec{\xi}^{p/l,l})^T , 
\end{equation}
and the $p \times p$ matrix $\vec{S}$ is defined as 
\begin{equation}%S
\vec{S}=
\left(
\begin{array}{ccc}
\vec{S}' & \cdots & 0        \\
         & \ddots &          \\
0        & \cdots & \vec{S}' \\
\end{array} 
\right) ,
\end{equation}
and the $l \times l$ matrix is defined as $\vec{S}'= (S_{\mu \nu}') = (\delta_{\mu, (\nu +1) \mod l})$. 
When $l=1$, i.e., $\vec{S}=\vec{1}$ ($\vec{1}$ is the unity matrix), the system is equivalent to the Hopfield model. 





\section{Path-integral analysis}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

D${\rm \ddot{u}}$ring et al discussed the sequential associative memory model by means of the path-integral analysis \cite{during1998}. 
In this section, we introduce macroscopic state equations for the model with a finite temperature $T \ge 0$, according to their paper. 

In order to analyze the transient dynamics, the generating function $Z[\vec{\psi}]$ is defined as 
%�܂��C���̐����֐�$Z[\vec{\psi}]$���`����D
%
\begin{equation}%(definition of generating functional Z[phi])
Z[\vec{\psi}]=\sum_{\vec{\sigma}(0),\cdots ,\vec{\sigma}(t)} p[\vec{\sigma}(0),\cdots ,\vec{\sigma}(t)] e^{-i\sum_{s<t}\vec{\sigma}(s)\cdot \vec{\psi}(s)}, 
\label{eq:def_Z}
\end{equation}
where $\vec{\psi}=(\psi(0),\cdots ,\psi(t-1))^T$ and the state $\vec{\sigma}(s)=(\sigma_1(s),\cdots ,\sigma_N(s))^T$ denotes the state of the spins at time $s$. 
%�����ŁC$\vec{\psi}=(\psi(0),\cdots ,\psi(t-1))^T$�ł���C$\vec{\sigma}(s)=(\sigma_1(s),\cdots ,\sigma_N(s))^T$�͎���$s$�ł̃l�b�g���[�N�̏�Ԃ������D
%
The probability $p[\vec{\sigma}(0),\cdots ,\vec{\sigma}(t)]$ denotes the probability of taking the path from initial state $\vec{\sigma}(0)$ to state $\vec{\sigma}(t)$ at time $t$ through $\vec{\sigma}(1),\vec{\sigma}(2),\cdots,\vec{\sigma}(t-1)$. 
%�܂��C$p[\vec{\sigma}(0),\cdots ,\vec{\sigma}(t)]$�͏������$\vec{\sigma}(0)$������$\vec{\sigma}(1),\vec{\sigma}(2),\cdots$���o�R���ď��$\vec{\sigma}(t)$�ɂȂ�m���������D
%
As (\ref{eq:def_Z}) shows, the generating functional entails the summation of all $2^{(t+1)N}$ paths which the system can take from time $0$ to $t$. 
%(\ref{eq:def_Z})��������킩��悤�ɁC�����֐��͎���$0$���玞��$t$�܂łɃl�b�g���[�N����蓾��$2^{(t+1)N}$�̌o�H�Ɋւ���a���Ƃ��Ă���D%���̂��߁C���̐����֐��ɂ���͕��@�͌o�H�ϕ��@�ƌĂ΂�Ă���D
%
One can obtain all the relevant order parameters, i.e., the overlap $m(s)$, the correlation function $C(s,s')$ and the response function $G(s,s')$, by calculating the appropriate derivatives of the above functional and letting $\vec{\psi}$ tend to $\vec{0}$ afterwards as follows: 
%�����֐�$Z[\vec{\psi}]$��胂�f���̋�����Ԃ�\���C�z�N���̃p�^�[���Ə�ԂƂ̏d�Ȃ��\���I�[�o���b�v$m(s)$�C��Ԃ̎��ԑ���$C(s,s')$�C��ԕω��̂��₷����\���ю���$G(s,s')$�����߂邱�Ƃ��ł���D
%
\begin{equation}%(definition of m(s))
m(s)=i \lim_{\vec{\psi}\to\vec{0}} \frac 1N \sum_{i=1}^N \xi^{1,s}_i \frac{\partial Z[\vec{\psi}]}{\partial \psi_i(s)}, 
\label{eq:def_m}
\end{equation}
\begin{equation}%(definition of C(s,s'))
C(s,s')=- \lim_{\vec{\psi}\to\vec{0}} \frac 1N \sum_{i=1}^N \frac{\partial^2 Z[\vec{\psi}]}{\partial \psi_i(s) \partial \psi_i(s')},
\label{eq:def_C}
\end{equation}
\begin{equation}%(definition of G(s,s'))
G(s,s')=i \lim_{\vec{\psi}\to\vec{0}} \frac 1N \sum_{i=1}^N \frac{\partial^2 Z[\vec{\psi}]}{\partial \psi_i(s)\partial \theta_i(s')}. 
\label{eq:def_G}
\end{equation}
Using the assumption of self-averaging, we replace the generating functional $Z[\vec{\psi}]$ with its disorder-averaged generating functional $\bar{Z}[\vec{\psi}]$. 
Evaluating the averaged generating function $\bar{Z}[\vec{\psi}]$ through the saddle point method, 
we obtain the following saddle-point equations for the order parameters of (\ref{eq:def_m})-(\ref{eq:def_G}) in the thermodynamical limit, i.e., $N\to\infty$ (See Appendix \ref{app:-1}). 
\begin{eqnarray}
m(s)    &=& \ll \xi \sigma(s) \gg , \label{eq:spe_m} \\
C(s,s') &=& \ll \sigma(s) \sigma (s') \gg , \label{eq:spe_C} \\
G(s,s') &=& \frac{\partial \ll \sigma(s) \gg}{\partial \theta(s')}. \label{eq:spe_G}
\end{eqnarray}
The average over the effective path measure is given by 
\begin{equation}
\ll g(\vec{\sigma},\vec{v}) \gg 
= \biggl< \int {\cal D}\vec{v} \Tr{\vec{\sigma}} g(\vec{\sigma},\vec{v}) p[\sigma(0)] \prod_{s=1}^{t} \frac 12 [1+\sigma(s)\tanh \beta h (\vec{\sigma},\vec{v},s-1)] \biggr>_{\vec{\xi}}, 
\label{eq:E_path}
\end{equation}
\begin{eqnarray}
{\cal D}\vec{v} &\equiv& \frac {d\vec{v} e^{-\frac 12\vec{v} \cdot \vec{R}^{-1} \vec{v}}} {\sqrt{(2\pi )^t|\vec{R}|}}, \\ 
\Tr{\vec{\sigma}} &\equiv& \sum_{\sigma(0) ,\cdots ,\sigma(t) \in \{ -1,1 \}}, \\
h(\vec{\sigma},\vec{v},s) &=& \xi ^{s+1} m(s)+\theta(s)+\sqrt{\alpha}v(s)+(\vec{\Gamma} \vec{\sigma})(s), \label{eq:local_field} \\
\vec{R} &=& \sum_{a=0}^{l-1} \sum_{m,n \ge 0}\vec{G}^{ml+a} \vec{C}(\vec{G}^\dagger)^{nl+a}, \label{eq:R} \\
\vec{\Gamma} &=& \frac \alpha l \sum_{\mu =0}^{l-1} e^{2\pi i \mu / l} [\vec{1}-e^{2\pi i \mu / l}\vec{G}]^{-1}, \label{eq:gamma} 
\end{eqnarray}
with $\vec{\Gamma}=\hat{\vec{K}}^\dagger$, $\hat{\vec{Q}}=-\frac 12 \alpha i \hat{\vec{R}}$ 
and $p[\sigma(0)]=\frac 12[1+\sigma(0)m(0)]$ which is the initial spin probability. 
The operator $<\cdot>_{\vec{\xi}}$ denotes the average over the condensed patterns. 
The term $(\vec{\Gamma} \vec{\sigma})(s)$ denotes the $s$th element of the vector $\vec{\Gamma} \vec{\sigma}$. 
The vectors $\vec{\sigma}$ and $\vec{v}$ denote $\vec{\sigma}=\{ \sigma(0),\cdots , \sigma(t) \}$ and $\vec{v}=\{ v(0),\cdots , v(t-1) \}$, respectively. 
Equations (\ref{eq:spe_m})-(\ref{eq:gamma}) entirely describe the dynamics of the system. 
The term $\prod_{s=1}^{t} \frac 12[1+\sigma(s)\tanh \beta h (\vec{\sigma},\vec{v},s-1)]$ in (\ref{eq:E_path}) cannot be factorized with respect to spin variables at different times. 
Calculation of the spin summation of (\ref{eq:E_path}) requires an exponential time $O(e^t)$ at time $t$. 
In the infinite-step sequence processing model, local field $h(\vec{\sigma},\vec{v},s)$ depends on only spin variables at time $s$ \cite{during1998}. 
Therefore the term $\prod_{s=1}^{t} \frac 12 [1+\sigma(s)\tanh \beta h (\vec{\sigma},\vec{v},s-1)]$ can be factorized, so the spin summations can be taken easily. 




\section{The stationary state}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In this section, we inspect time-translation invariant solutions of our macroscopic equations (\ref{eq:spe_m})-(\ref{eq:spe_G}) for the deterministic dynamics, i.e., $\beta\to\infty$ ($T=0$). 
The time-translation invariant solutions will describe motion on a macroscopic limit cycle: 
\begin{equation}
\left\{
\begin{array}{l}
m(s)=m, \\
C(s,s')=C(s-s'), \\
G(s,s')=G(s-s'), \\
\end{array}
\right.
\label{eq:condition}
\end{equation}
with $\theta(s)=\theta$. 
Now, we disregard the transient states. %---ADDED---
Note that the condition of (\ref{eq:condition}) includes an unspoken condition that the transient states are disregarded. %---ADDED---
Therefore, we put that the dynamics is already in the stationary state at time $s=0$ under this assumption. 
In the zero noise limit, i.e., $T=0$ ($\beta\to\infty$), the dynamics becomes deterministic. 
Therefore, we also assume that the system takes a fixed path as 
%---DELETED---In the stationary state, the system takes the same states repeatedly; therefore the dynamics of the system takes a fixed path. 
%�܂�C����Ԃł��鎞��$s \ge \tau$�ł́C�o�H��1�Ɍ��肳���D
%
%---DELETED---Namely, the time $\tau$, which satisfies following relationships, exists: 
%����Ԃł́C�n�̏�Ԃ͎���$l$�œ�����Ԃ��J��Ԃ��Ă��邽�߁C...�ƂȂ鎞��$s=\tau$�����݂���D
%
\begin{equation}
\sigma(s+l)=\sigma(s),
\end{equation}
for any time $s \ge 0$. 
The path which the system takes after time $s\ge 0$ can be described as 
\begin{equation}
\sigma(s)=\eta_s, \label{eq:stationary_state}
\end{equation}
by only $l$ constants $\eta_0,\cdots,\eta_{l-1}\in\{-1,1\}$. 
%$\sigma(\tau)=\sigma$�Ƃ���ƁC$s\ge\tau$�ȍ~�̌o�H�͒萔$\eta_0,\cdots,\eta_{l-1}\in\{-1,+1\}$�ɂ���āC$\sigma(s+\tau')=\eta_{\tau'}\sigma$�Ƃ������Ƃ��ł���D
%
The pattern index $s$ of the constants $\eta_s$ is understood to be taken modulo $l$. 
%�����ŁC$\eta_s$�̓Y������ $\mod l$���Ƃ�Ƃ���D
%
Note that it is not necessary to calculate these constants $\{\eta_s\}$ explicitly. 
%���̒萔�����߂�K�v�͂Ȃ����Ƃɒ��ӂ��ꂽ���D
%
When the variable transformation 
\begin{equation}
\chi (s) = \eta_s \sigma (s), 
\label{eq:transformation}
\end{equation}
is carried out to spin variables $\sigma (s)$, 
the transformed spin variables $\chi(s)$ take same value for any time $s$, i.e., $\chi(s)=\chi(s')$ for any $s,s'$. 
Equation (\ref{eq:E_path}) means the expectation of $g(\vec{\sigma},\vec{v})$ with respect to the path probability. 
%(\ref{eq:E_path})���̒��̍�$\Tr{\vec{\sigma}}$�́C$g(\vec{\sigma},\vec{v})$�� path probability �Ɋւ�����Ғl�ł���C
%
%---DELETED---In the zero noise limit, the dynamics defined by (\ref{eq:dynamics}) is equivalent to $\sigma_i(t+1)=\sgn h_i(t)$. 
%����_�I�_�C�i�~�N�X�̂Ƃ��C(\ref{eq:dynamics})���̃_�C�i�~�N�X�́C$\sigma_i(t+1)=\sgn h_i(t)$�Ɠ����ł���D
%
In the zero noise limit, equation (\ref{eq:E_path}) becomes 
\begin{equation}
\ll g(\vec{\sigma},\vec{v}) \gg 
= \biggl< \int {\cal D} \vec{v} \Tr{\vec{\sigma}} g(\vec{\sigma},\vec{v}) p[\sigma(0)] \prod_{s=1}^t \delta_{\sigma(s), \sgn h (\vec{\sigma},\vec{v},s-1)} \biggr>_{\vec{\xi}}, 
\label{eq:E_path2}
\end{equation}
%Note that the spin variables are deterministic 
%even if (\ref{eq:E_path2}) includes the Gaussian random fields $\vec{v}$. 
When the spin variables have periodicity as $\sigma(s+l)=\sigma(s)$, 
the Gaussian random fields are also deterministic as $v(s+l)=v(s)$ (See Appendix \ref{app:0}). 
%Hence, the spin variable evolves deterministically. 
For any function $\phi(\{\sigma(s)\})$ and any constants $c_0, \cdots ,c_{t} \in \{ -1,1 \}$, the following identity holds: 
\begin{equation}
\sum_{\sigma (0), \cdots ,\sigma (t) \in \{ -1,1 \}} \phi(\sigma(0), \cdots ,\sigma(t))
=\sum_{\sigma (0), \cdots ,\sigma (t) \in \{ -1,1 \}}  \phi(c_0 \sigma(0), \cdots , c_t \sigma(t)). 
\label{eq:const}
\end{equation}
Applying (\ref{eq:const}) to (\ref{eq:E_path2}) and substituting (\ref{eq:transformation}), we obtain 
\begin{eqnarray}
& & \ll g(\vec{\sigma},\vec{v}) \gg \nonumber \\
&=& \biggl< \int {\cal D} \vec{v} \Tr{\vec{\sigma}} g(\{c_0\sigma(0),\cdots,c_t\sigma(t)\},\vec{v}) p[c_0\sigma(0)] \prod_{s=1}^t \delta_{c_s\sigma(s), \sgn h (\{c_0\sigma(0),\cdots,c_t\sigma(t)\},\vec{v},s-1)} \biggr>_{\vec{\xi}} \nonumber \\
&=& \biggl< \int {\cal D} \vec{v} \Tr{\vec{\chi}} g(\{c_0\eta_0\chi (0),\cdots,c_t\eta_t\chi (t)\},\vec{v}) \nonumber \\
& & \qquad \times p[c_0\eta_0\chi (0)] \prod_{s=1}^t \delta_{c_s\eta_s\chi (s), \sgn h (\{c_0\eta_0\chi (0),\cdots,c_t\eta_t\chi (t)\},\vec{v},s-1)} \biggr>_{\vec{\xi}}, \nonumber \\
&=& \biggl< \int {\cal D} \vec{v} \Tr{\vec{\chi}} g(\{\chi (0),\cdots,\chi (t)\},\vec{v}) p[\chi (0)] \prod_{s=1}^t \delta_{\chi (s), \sgn h (\{\chi (0),\cdots,\chi (t)\},\vec{v},s-1)} \biggr>_{\vec{\xi}}, 
\label{eq:E_path3}
\end{eqnarray}
with 
\begin{equation}
\Tr{\vec{\chi}} \equiv \sum_{\eta_0 \chi(0), \cdots , \eta_t \chi (t) \in \{ -1,1 \} }
= \sum_{\chi(0), \cdots , \chi (t) \in \{ -1,1 \} }.
\end{equation}
In derivation of (\ref{eq:E_path3}), we put the constants $\{c_s\}$ as $c_s = \eta_s$ for all $s$. 
Generality is kept even if $c_s = \eta_s$ for all $s$. 
Namely, with respect to the transformed spin variable $\chi(s)$, 
the effective single spin described by (\ref{eq:E_path3}) is 
\begin{equation}
\chi (s) = \sgn h (\{\chi (0),\cdots,\chi (t)\},\vec{v},s-1). 
\label{eq:ess}
\end{equation}
The transformed spin variables $\chi(s)$ are deterministic 
even if (\ref{eq:ess}) includes the Gaussian random fields $\vec{v}$, 
since the Gaussian random fields are deterministic. 
In order to get rid of the self-interaction, we assume the Maxwell construction ansatz. 
Using the identity $\chi(s)=\chi(s')$ for any $s,s'$ to (\ref{eq:ess}) and applying the Maxwell construction, we get 
\begin{eqnarray}
\chi (s) 
&=& \sgn [\xi^s m(s-1)+\theta(s-1)+\sqrt{\alpha}v(s-1)+ \sum_{s'<s} \Gamma(s,s') \chi(s')] \nonumber \\
&=& \sgn [\xi^s m(s-1)+\theta(s-1)+\sqrt{\alpha}v(s-1)+ \chi (s) \sum_{s'<s} \Gamma(s,s')] \nonumber \\
&=& \sgn h(\vec{v},s-1), \label{eq:Maxwell}
\end{eqnarray}
with $h(\vec{v},s) \equiv \xi^{s+1} m(s)+\theta(s)+\sqrt{\alpha}v(s)$. 
Substituting (\ref{eq:Maxwell}) into (\ref{eq:E_path3}), we obtain 
\begin{equation}
\ll g(\vec{\sigma},\vec{v}) \gg 
= \biggl< \int {\cal D} \vec{v} \Tr{\vec{\chi}} g(\vec{\chi},\vec{v}) p[\chi (0)] \prod_{s=1}^t \delta_{\vec{\chi}, \sgn h (\vec{v},s-1)} \biggr>_{\vec{\xi}}. 
\label{eq:E_path4}
\end{equation}
Thus, we can get rid of the self-interaction in the single spin problem by using the Maxwell construction in the zero noise limit, i.e., $T=0$. 
Since (\ref{eq:E_path4}) can be factorized with respect to the tranformed spin variables $\chi(s)$ at different times, we can easily perform the spin summations. 
%(\ref{eq:pp})���͎����Ɋւ��Ĉ��������ł���̂ŁCspin summation�͊ȒP�Ɍv�Z�ł���D
%
After simple rescalings we arrive at 
%�ȒP�ȕϐ��ϊ��ɂ��(\ref{eq:spe_m})-(\ref{eq:spe_G})���͎��̂悤�ɂȂ�D
%
\begin{eqnarray}
m(s) &=& \biggl< \xi^s \int {\cal D} \vec{v} \sgn h(\vec{v},s-1) \biggr>_{\vec{\xi}}, \label{eq:m2} \\
C(s,s') &=& \delta_{s,s'} +[1-\delta_{s,s'}] \biggl< \int {\cal D} \vec{v} [\sgn h(\vec{v},s-1)] [\sgn h(\vec{v},s'-1)] \biggr>_{\vec{\xi}}, \\
G(s,s') &=& \delta_{s,s'-1} \lim_{\beta \to \infty} \beta \biggl\{ 1 - \biggl< \int {\cal D} \vec{v} \tanh^2 \beta h(\vec{v},s-1) \biggl>_{\vec{\xi}} \biggr\} . \label{eq:G2}
\end{eqnarray}
%with $\{d\vec{v}d\vec{w}\}=\prod_{s<t}\frac{dv(s)dw(s)}{2\pi}$. 
%
%Note that we can also obtain (\ref{eq:m2})-(\ref{eq:G2}) from (\ref{eq:E_path4_dash}). 
We now calculate the matrix $\vec{R}$ under the condition of (\ref{eq:condition}). 
%(\ref{eq:condition})���̏����̂Ƃ���$\vec{R}$���v�Z����D
%
Since the matrices $\vec{G}$ and $\vec{C}$ become Toeplitz matrices (especially $\vec{C}$ is symmetric) under this conditions, 
$\vec{C}$ and $\vec{G}$ can be approximately regarded as commuting matrices, i.e., $\vec{C}\vec{G}=\vec{G}\vec{C}$. 
%���̂Ƃ��C$\vec{G}$�̓e�v���b�c�s��C$\vec{C}$�͑Ώ̃e�v���b�c�s��ƂȂ邽�߁C$\vec{C}$��$\vec{G}$�͋ߎ��I�ɉ�$\vec{C}\vec{G}=\vec{G}\vec{C}$�ƂȂ�D
%
Therefore, the matrix $\vec{R}$ simplifies to 
%�܂��C(\ref{eq:R})����$\vec{R}=(R(s,s'))$��$R(s,s')=R(s-s')$�Ƃ������Ƃ��ł���D
%
\begin{eqnarray}
\vec{R}
&=& \sum_{a=0}^{l-1} \vec{G}^a \biggl( \sum_{m,n\ge 0} \vec{G}^{ml} (\vec{G}^\dagger)^{nl} \vec{C} \biggr) (\vec{G}^\dagger)^a \nonumber \\
&=& \sum_{a=0}^{l-1} \vec{G}^a \biggl( [\vec{1}-\vec{G}^l]^{-1} [\vec{1}-(\vec{G}^\dagger)^l]^{-1} \vec{C} \biggr) (\vec{G}^\dagger)^a \nonumber \\
&=& [\vec{1}-\vec{G}\vec{G}^\dagger]^{-1}[\vec{1}-(\vec{G}\vec{G}^\dagger)^l] [\vec{1}-\vec{G}^l]^{-1} [\vec{1}-(\vec{G}^\dagger)^l]^{-1} \vec{C}, 
\label{eq:R}
\end{eqnarray}
We consider the persistent parts of $C(\tau)$ and $R(\tau)$ as $C(\tau)\to q$ and $R(\tau)\to r$ for $\tau\to\infty$ and also consider the non-persistent parts $\tilde{C}(\tau)\to 0$ and $\tilde{R}(\tau)\to 0$, i.e., $C(\tau)=q+\tilde{C}(\tau)$ and $R(\tau)=r+\tilde{R}(\tau)$. 
%$\tau\to\infty$��$C(\tau)\to q$,$R(\tau)\to r$�Ɏ�������Ƃ��āC$C(\tau)=q+\tilde{C}(\tau)$,$R(\tau)=r+\tilde{R}(\tau)$�Ƃ����D
%
Upon rewriting $G(\tau)=\beta\delta_{\tau,1}[1-\tilde{q}]$ and $r=q\rho$ given by (\ref{eq:R}), we obtained 
%���̂Ƃ��C(\ref{eq:R})�����$r=q\rho$�Ƃ�����D�\�L���ȒP�ɂ��邽�߂ɁC$G(\tau)=\beta\delta_{\tau,1}[1-\tilde{q}]$�Ƃ����ƁC(\ref{eq:m2})-(\ref{eq:G2})���͎��̂悤�ɂȂ�D
%
\begin{eqnarray}
m
&=& \biggl< \xi \int Dz \int \tilde{{\cal D}} \vec{v} \sgn h(\vec{v},0|z) \biggr>_{\vec{\xi}}, \\
q
&=& \biggl< \int Dz \int \tilde{{\cal D}} \vec{v} [\sgn h(\vec{v},\tau|z)][\sgn h(\vec{v},0|z)]\biggr>_{\vec{\xi}}, \\
\tilde{q}
&=& \lim_{\beta\to\infty} \biggl< \int Dz \int \tilde{{\cal D}} \vec{v} \tanh^2 \beta h(\vec{v},0|z) \biggr>_{\vec{\xi}}, \\
r
&=& q\rho,
\end{eqnarray}
from (\ref{eq:m2})-(\ref{eq:G2}) 
with $\tilde{{\cal D}} \vec{v} \equiv d\vec{v} e^{- \frac 12 \vec{v} \cdot \tilde{\vec{R}}^{-1} \vec{v}} [(2\pi )^t|\tilde{\vec{R}}|]^{-1/2}$, 
$Dz=\frac{dz}{\sqrt{2\pi}}e^{-z^2/2}$ and $h(\vec{v},\tau|z)\equiv \xi^\tau m +\theta + z\sqrt{\alpha q \rho}+\sqrt{\alpha}v(\tau)$. 
%�����ŁC$Dz=\frac{dz}{\sqrt{2\pi}}e^{-z^2/2}$�ł���D
%
The matrix $\vec{G}$ is given by 
%(\ref{eq:G2})�����C$\vec{G}$�͎��̂悤�ȍs��ł��邱�Ƃ��킩��D
%
\begin{equation}
\vec{G}=
\left(
\begin{array}{ccccc}
0      & 0      & 0      & \cdots & 0      \\
G(1)   & 0      & 0      & \cdots & 0      \\
0      & G(1)   & 0      & \cdots & 0      \\
\vdots &        & \ddots &        & \vdots \\
0      & \cdots & 0      & G(1)   & 0      \\
\end{array}
\right) .
\end{equation}
from (\ref{eq:G2}). 
Equation (\ref{eq:R}) can be changed to 
%(\ref{eq:R})���͎��̂悤�ɕό`�ł���D
%
\begin{equation}
[\vec{1}-(\vec{G}^\dagger)^l-\vec{G}^l-(\vec{G}^\dagger\vec{G})^l] [\vec{1}-(\vec{G}\vec{G}^\dagger)^l]^{-1} [\vec{1}-\vec{G}\vec{G}^\dagger] \vec{R}=\vec{C}. 
\label{eq:R_modified}
\end{equation}
The identity $\vec{G}\vec{G}^\dagger \simeq G(1)^2\vec{1}$ holds in the case where time $s$ is sufficiently large, so the left-hand side of (\ref{eq:R_modified}) becomes 
%����$t$���傫���Ƃ��C$\vec{G}\vec{G}^\dagger \simeq G(1)^2\vec{1}$�Ƌߎ��ł���̂ŁC(\ref{eq:R_modified})���̍��ӂ͎��̂悤�ɂȂ�D
%
\begin{eqnarray}
& & [\vec{1}-(\vec{G}^\dagger)^l-\vec{G}^l-(\vec{G}^\dagger\vec{G})^l] [\vec{1}-(\vec{G}\vec{G}^\dagger)^l]^{-1} [\vec{1}-\vec{G}\vec{G}^\dagger] \vec{R} \nonumber \\
&=& \left(
\begin{array}{cccc}
D(0)   & D(1)   & \cdots & D(t-1) \\
D(1)   & D(0)   & \cdots & D(t-2) \\
\vdots & \vdots & \ddots & \vdots \\
D(t-1) & D(t-2) & \cdots & D(0)   \\
\end{array}
\right) ,
\end{eqnarray}
where 
%�������C
%
\begin{eqnarray}
D(0) &=& (1+g_1^2)g_2R(0)   -2g_1g_2R(1), \\
D(s) &=& (1+g_1^2)g_2R(s)- g_1g_2[R(s-1)+R(s+1)], \; \; (0<s<t-1)\\
D(t-1) &=& (1+g_1^2)g_2R(t-1) -2g_1g_2R(t-2), 
\end{eqnarray}
with $g_1 \equiv G(1)^l$ and $g_2 \equiv [1-G(1)^2]/[1-G(1)^{2l}]$. 
Since $\vec{D}$ and $\vec{C}$ are symmetric Toeplitz matrices, they can be diagonalized by using the discrete Fourier transformation (See Appendix \ref{app:1}). 
%�s��$\vec{D},\vec{C}$�͑Ώ̃e�v���b�c�s��Ȃ̂ŁC�t�[���G�ϊ��ɂ���đΊp���\�ł���D
%
The Fourier transformations (or the Lattice Green's functions) $\hat{D}_k,\hat{C}_k$ of the matrices $\vec{D},\vec{C}$ are given by 
%$\vec{D},\vec{C}$�̃t�[���G�ϊ��i�i�q�O���[���֐��j$\hat{D}_k,\hat{C}_k$�͎��̂悤�ɂȂ�D
%
\begin{eqnarray}
\hat{D}_k &\simeq& \sum_{\tau=0}^{t-1} \biggl\{ (1+g_1^2)g_2R(\tau)-g_1g_2[R(\tau-1)-R(\tau+1)] \biggr\} e^{ik\tau}, \\
\hat{C}_k &=& \sum_{\tau=0}^{t-1} C(\tau)e^{ik\tau}. 
\end{eqnarray}
For any wave number $k$, $\hat{D}_k=\hat{C}_k$ holds when $\vec{D}=\vec{C}$. 
%$\vec{D}=\vec{C}$, �̂Ƃ��C�C�ӂ̔g��$k$�ɂ���$\hat{D}_k=\hat{C}_k$�ƂȂ��Ă���D
%
Taking the limit $s\to\infty$ about $\hat{D}_0=\hat{C}_0$, the following relationship is obtained: 
%$k=0$�̏ꍇ�ɂ���$t\to\infty$�̋Ɍ����Ƃ邱�Ƃɂ���āC
%
\begin{equation}
r = \frac{1-G(1)^{2l}}{(1-G(1)^2)(1-G(1)^l)^2} q. 
\label{eq:r}
\end{equation}
By working out the remaining integrals over $\vec{v}$ and setting $\theta=0$, 
we finally obtain the stationary state equations of the order parameters as follows: 
\begin{eqnarray}
m &=& {\rm erf} \biggl( \frac m{\sqrt{2\alpha \rho}} \biggr), \label{eq:sna_m} \\
U &=& \sqrt{\frac 2{\pi\alpha \rho}}e^{-\frac {m^2}{2\alpha \rho}}, \\
\rho &=& \frac{1-U^{2l}}{(1-U^2)(1-U^l)^2}, \label{eq:sna_rho}
\end{eqnarray}
with $q=1$, $\tilde{q}=1$ and $U\equiv G(1)$ where ${\rm erf}(\cdot)$ denotes Error function defined by ${\rm erf}(x) \equiv \frac 2\pi \int_0^x e^{-u^2} du$. 
%�܂��C$q=1$,$\tilde{q}=1$�ł���C�����ŁC$U=\lim_{\beta\to\infty}G(1)$�ł���D�����ŁCerf ��${\rm erf}(x) \equiv \frac 2\pi \int_0^x e^{-u^2} du$ �Œ�`�����덷�֐��ł���D
%
It turns out that these stationary state equations (\ref{eq:sna_m})-(\ref{eq:sna_rho}) given by this exact solution are equivalent 
to those of the signal-to-noise analysis (See Appendix \ref{app:2}) \cite{mimura1995}. 
%����́C�o�͊֐��𕄍��֐��Ƃ����Ƃ��́C���~�b�g�T�C�N�������A�z�L�����f����SCSNA�ɂ���͌��ʂƈ�v����(See Appendix \ref{app:2})�D
%
Figure \ref{fig:fig1} shows that the storage capacity $\alpha_c$ and the number of patterns per one limit cycle $s$. 
%�L���e�ʂ̃��~�b�g�T�C�N����$l$�ˑ�����}\ref{fig:fig1}�Ɏ����D
%
Figures \ref{fig:fig2} and \ref{fig:fig3} compare the theoretical results and computer simulations for $l=3,7$ (The number of spins is $N=3000$, the number of iterations is $11$). 
The data points and error bars show the results of the computer simulation. 
With respect to the computer simulation in figure \ref{fig:fig2} and \ref{fig:fig3}, 
the stationary overlaps are defined as $m(100l)$ and $m(50l)$, respectively. 
%�}\ref{fig:fig2},\ref{fig:fig3}�ɃI�[�o���b�v�̋L�����ˑ����ɂ��āC���_�l�ƌv�Z�@�����Ƃ̔�r�������ʂ������D
%
It is confirmed that the theoretical results are in good agreement with the computer simulations. 
%���_�͎����Ƃ悭��v���Ă��邱�Ƃ��킩��D
%
Storage capacity monotonously increases from $\alpha_c=0.138$ ($l=1$) with the number of steps $l$ . 
%
In the large $l$ limit, storage capacity finally converges to $\alpha_c=0.269$, which coincides with the theoretical result of for the infinite-step sequence processing model given by D${\rm \ddot{u}}$ring et al \cite{during1998}. 
%
The original properties of the finite-step sequence processing model appear as long as the number of steps of a limit cycle $l$ has order $l=O(1)$. 
%�܂��A$s=O(1)$ �̃I�[�_�ł̂ݗL���X�e�b�v�̐����������Ă���A
%
In the case that $l$ has the order more than $O(1)$, the properties are the same as the properties of the infinite-step sequence processing model. 
%����ȏ�̃I�[�_�ł͖����X�e�b�v�Ɠ��������������Ƃ��킩�����B
%
\begin{figure}[htbp]
\begin{center}
\includegraphics[width=.6\linewidth,keepaspectratio]{fig1.eps}
\caption{the storage capacity $\alpha_c$ and the number of patterns per one limit cycle $s$.}
\label{fig:fig1}
\end{center}
\end{figure}
%
\begin{figure}[htbp]
\begin{center}
\includegraphics[width=.6\linewidth,keepaspectratio]{fig2.eps}
\caption{Computer simulations ($l=3$,$N=3000$, $11$ times): the overlap $m$ and loading rate $\alpha$.}
\label{fig:fig2}
\end{center}
\end{figure}
%
\begin{figure}[htbp]
\begin{center}
\includegraphics[width=.6\linewidth,keepaspectratio]{fig3.eps}
\caption{Computer simulations ($l=7$, $N=3000$, $11$ times): the overlap $m$ and loading rate $\alpha$.}
\label{fig:fig3}
\end{center}
\end{figure}








\section{Conclusions}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We exactly analyzed an associative memory model storing an infinite number of limit cycles with finite steps by means of the path-integral method. 
%�L���X�e�b�v�n��z�N���f���̑z�N�ߒ����o�H�ϕ��ɂ���Č����ɉ�͂����D
%
In the case where the dynamics is deterministic, 
the statistical properties are simplified like those of the infinite-step sequence processing model by using Maxwell construction. 
%���x$T=0$�̌���_�I�_�C�i�~�N�X�ł̏�ԑJ�ڂ̏ꍇ�ɂ��ẮC��ԑJ�ڂ̎������𗘗p����ƁC
%�e�����ł̏o�͂����C�ߋ��̏�Ԃւ̈ˑ����������X�e�b�v�n��z�N���f���Ɠ��l�ɊȒP�ɂȂ邱�Ƃ��킩�����D
%
We also derived the macroscopic equations for stationary states at $T=0$. 
%���x$T=0$�̏ꍇ�ɂ��āC����Ԃ��L�q���鋐����ԕ�������z�N�ߒ����瓱�o�����D
%
\par
We obtained the dependence of storage capacity ($\alpha_c$) on the number of patterns per one limit cycle ($l$). 
%����Ԃ��L�q���鋐����ԕ���������C�n��̃X�e�b�v���ƋL���e�ʂ̊֌W�����߂��D
%
At $l=1$, storage capacity is $\alpha_c=0.138$, as in the Hopfield model. 
%�X�e�b�v����$l=1$�̂Ƃ��́C�L���e�ʂ�Hopfield���f���Ɠ���$\alpha_c=0.138$�ƂȂ�D
%
A storage capacity monotonously increases with the number of limit cycles, and converges to $\alpha_c=0.269$ at $l\simeq 10$. 
%�X�e�b�v���̑����ƂƂ��ɒP���ɋL���e�ʂ͑������C
%�X�e�b�v��$l=10$���x�ŁC�����ɖ����X�e�b�v�n��z�N���f���̋L���e��$\alpha_c=0.269$�Ɏ�������D
%
The original properties of the finite-step sequence processing model appear as long as the number of steps of the limit cycle has order $l=O(1)$. 
%�X�e�b�v��$l=O(1)$�ł̂ݗL���X�e�b�v�ɓ��L�̐����������Ƃ��킩�����D
%
\par
%According to Bolle et al, the transient dynamics given by signal-to-noise analysis is exact up to only time step 3 and inexact to step 4 or beyond. 
%Bolle��̕񍐂ɂ��ƁC
%S/N��͂ɂ��z�N�ߒ��̉�͌��ʂ́C�����̐��X�e�b�v�̉ߓn�����ɂ��Ă��������ƂȂ��Ă���D
%
In the case where the dynamics is deterministic, we also derived the stationary state equations by using Maxwell construction. 
The stationary state equations are equivalent to those of the signal-to-noise analysis. 
%�܂��C���̒���Ԃ��L�q���鋐����ԕ������́C�ȑO��X�����߂�S/N��͂ɂ���͌��ʂƈ�v�����D
%
%---DELETED---The path-integral method, which does not require a Gaussian approximation for crosstalk noise, 
%---DELETED---and the signal-to-noise analysis, which needs such an approximation, gave the same results in the stationary state. 
%���̂悤�ɁCS/N��͂ł̓N���X�g�[�N�m�C�Y���K�E�X���z�ɏ]���Ɖ��肷��ɂ�������炸�C
%����Ԃɂ����Ă̓K�E�X�ߎ����s�K�v�Ȍo�H�ϕ��ɂ���͌��ʂƈ�v�����D
%
This means that the signal-to-noise analysis applied to the stationary state is exact 
in spite of including errors in the middle of the transient dynamics in the zero noise limit. 
%�܂�CS/N��͂ɂ�����Ԃ̉�͌��ʂ��������ł��邱�Ƃ��킩�����D
%
%---DELETED---\par
%---DELETED---In the finite-step sequence processing model, 
%---DELETED---although the signal-to-noise analysis applied to the transient dynamics includes errors in the middle of the transitional state, 
%---DELETED---the signal-to-noise analysis can give exact solutions in both the stationary state and the first few time steps of the transitional dynamics in the case where $T=0$. 
%����ɂ��C���v�_�o�͊w�Ȃǂ�S/N��͂ɂ�鎩�ȑz�N���f����L���X�e�b�v�n��z�N���f���̓������̉�͂́C
%�z�N�ߒ��̓r���̒i�K�ł͌덷���܂ނ��̂́C�����̐��X�e�b�v�ƒ���ԂɊւ��Ă͌o�H�ϕ��Ɠ��l�Ɍ����ł��邱�Ƃ��킩��D
%












\begin{center}
{\small \bf Acknowledgement}
\end{center}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This work was partially supported by a Grant-in-Aid 
for Scientific Research on Priority Areas No. 14084212, 
for Scientific Research (C) No. 14580438, 
and for Encouragement of Young Scientists (B) No. 15700141 
from the Ministry of Education, Culture, Sports, Science and Technology of Japan. 







\appendix

\section{Derivation of order parameter equations\label{app:-1}}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

D${\rm \ddot{u}}$ring et al discussed the sequential associative memory model by using the path-integral method \cite{during1998}. 
%D${\rm \ddot{u}}$ring��́C�n��z�N���f�����o�H�ϕ��@�ɂ���ċc�_���Ă���D
%
Here , we discuss for the model with finite temperature $T$, according to their paper. 
%�����ł́C�ނ�ɏ]���L�����x���f���ɑ΂��鋐����ԕ������������D���̎��̓��o�̏ڍׂ͕���\cite{during1998}���Q�Ƃ��ꂽ���D�o�H�ϕ��@�ɂ���ĉ�͂���\cite{during1998,kawamura2002}�D
%
Most of the technical detail to derive order parameter equations is almost identical to the paper of D${\rm \ddot{u}}$ring et al \cite{during1998}. %---ADDED---
%������ԕ������̓��o���@��D"uring�̘_���ƂقƂ�Ǔ����ł��D
%
%The detailed derivation is available in their paper \cite{during1998}. 
%�z�N�ߒ����L�q���鋐���I��ԕ������𓱏o���āC���̒���Ԃ��c�_����D
%


%---DELETED---\subsection{Derivation of saddle-point equations}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

%The dynamics (\ref{eq:dynamics}) is a Markov chain, so the path probabilities $p[\vec{\sigma}(0),\cdots ,\vec{\sigma}(t)]$ are simply given by products of the individual transition probabilities $W[\vec{\sigma}(s+1)|\vec{\sigma}(s)]$, i.e., $p[\vec{\sigma}(0),\cdots ,\vec{\sigma}(t)] = p[\vec{\sigma}(0)]\prod_{s=0}^{t-1}W[\vec{\sigma}(s+1)|\vec{\sigma}(s)]$.  %<----------(\ref{eq:dynamics})���̃_�C�i�~�N�X�̓}���R�t�A���ł��邽�߁Cpath probability p �͌X�̏�ԑJ�ڊm�� w �̐ςŗ^������D
The generating functional $\bar{Z}[\vec{\psi}]$ contains both condensed and non-condensed patterns. 
We isolate the non-condensed ones by introducing the local field $\vec{h}$ and the variables $\vec{x},\vec{y}$: %<----------�����֐��Ƀf���^�֐��̐ϕ��`�C$\delta (x) = \int_{-\infty}^\infty \frac{d\hat{x}}{2\pi} e^{i\hat{x}x}$��p���ĕ\�����Đ����֐��Ɏ�荞�ށD%�܂��C���鎞���ɂ͂P�̃p�^�[���݂̂��z�N����Ă���Ƃ��āC��z�N�p�^�[���Ɋւ���⏕�ϐ�$x_{\nu\mu}(s),y_{\nu\mu}(s)$���f���^�֐����g���ē�������D
\begin{eqnarray}%(introduction of h,x,y)
1 &=& \int \frac{d\vec{h} d\hat{\vec{h}}}{(2\pi)^{Nt}} \prod_i e^{i \hat{h}_i(s) \bigl[ h_i(s)-\sum_j J_{ij} \sigma_j(s) - \theta_i(s) \bigr] } , \label{eq:def_h} \\
1 &=& \int \frac{d\vec{x} d\hat{\vec{x}}}{(2\pi)^{(p-1)t}} e^{i \sum_{s<t} \sum_{\nu =1}^{p/l} \sum_{\mu =1 (\mu \ne s \; {\rm at} \; \nu=1)}^{l} \hat{x}_{\nu\mu}(s) \bigl[ x_{\nu\mu}(s) - \frac 1{\sqrt{N}} \sum_i \xi_i^{\nu,\mu +1} \hat{h}_i(s) \bigr]} , \\
1 &=& \int \frac{d\vec{y} d\hat{\vec{y}}}{(2\pi)^{(p-1)t}} e^{i \sum_{s<t} \sum_{\nu =1}^{p/l} \sum_{\mu =1 (\mu \ne s \; {\rm at} \; \nu=1)}^{l} \hat{y}_{\nu\mu}(s) \bigl[ y_{\nu\mu}(s) - \frac 1{\sqrt{N}} \sum_i \xi_i^{\nu,\mu}    \sigma_i(s)  \bigr]} . 
\end{eqnarray}
We isolate the various relevant macroscopic observables by inserting integrals over appropriate $\delta$-functions: %<----------����ɁC�����ϐ����ȉ��̂悤�ɓ�������D
\begin{eqnarray}%(introduction of m,k,q,Q,K)
1 &=& \int \frac{d\vec{m} d\hat{\vec{m}}}{(2\pi /N)^t}     e^{i \sum_{s<t}    \hat{m}(s)    \bigl[ m(s)    - \frac 1N \sum_i \xi_i^{1,s}   \sigma_i(s)   \bigr]} , \\
1 &=& \int \frac{d\vec{k} d\hat{\vec{k}}}{(2\pi /N)^t}     e^{i \sum_{s<t}    \hat{k}(s)    \bigl[ k(s)    - \frac 1N \sum_i \xi_i^{1,s+1} \hat{h}_i(s)  \bigr]} , \\
1 &=& \int \frac{d\vec{q} d\hat{\vec{q}}}{(2\pi /N)^{t^2}} e^{i \sum_{s,s'<t} \hat{q}(s,s') \bigl[ q(s,s') - \frac 1N \sum_i \sigma_i(s)   \sigma_i(s')  \bigr]} , \\
1 &=& \int \frac{d\vec{Q} d\hat{\vec{Q}}}{(2\pi /N)^{t^2}} e^{i \sum_{s,s'<t} \hat{Q}(s,s') \bigl[ Q(s,s') - \frac 1N \sum_i \hat{h}_i(s)  \hat{h}_i(s') \bigr]} , \\
1 &=& \int \frac{d\vec{K} d\hat{\vec{K}}}{(2\pi /N)^{t^2}} e^{i \sum_{s,s'<t} \hat{K}(s,s') \bigl[ K(s,s') - \frac 1N \sum_i \sigma_i(s)   \hat{h}_i(s') \bigr]} . \label{eq:def_K}
\end{eqnarray}
The generating functional which for $N\to\infty$ will be dominated by saddle points. 
We obtain 
\begin{equation}
\bar{Z}[\vec{\psi}] = \int d\vec{m}d\hat{\vec{m}} d\vec{k}d\hat{\vec{k}} d\vec{q}d\hat{\vec{q}} d\vec{Q}d\hat{\vec{Q}} d\vec{K}d\hat{\vec{K}} e^{N(\Psi + \Phi + \Omega)+{\cal O}(N^{1/2})}, 
\label{eq:Zbar}
\end{equation}
by substituting (\ref{eq:def_h})-(\ref{eq:def_K}) into (\ref{eq:def_Z}), where 
\begin{eqnarray}
\Psi 
&=& i \sum_{s<t} \biggl[ \hat{m}(s)m(s) + \hat{k}(s)k(s) - m(s)k(s) \biggr] \nonumber \\
& & + i \sum_{s,s'<t} \biggl[ \hat{q}(s,s')q(s,s') + \hat{Q}(s,s')Q(s,s') + \hat{K}(s,s')K(s,s') \biggr] , \label{eq:Psi} \\
\Phi
&=& \frac 1N \sum_i \ln \Tr{\vec{\sigma}} p_i(\sigma(0)) \int \{ d\vec{h} d\hat{\vec{h}} \} \nonumber \\
& & \times e^{\sum_{s<t} \bigl[ \beta \sigma(s+1) - \ln 2 \cosh \beta h(s) \bigr] } \nonumber \\
& & \times e^{- i \sum_{s,s'<t} \bigl[ \hat{q}(s,s')\sigma(s)\sigma(s') + \hat{Q}(s,s')\hat{h}(s)\hat{h}(s') + \hat{K}(s,s')\sigma(s)\hat{h}(s') \bigr] } \nonumber \\
& & \times e^{i \sum_{s<t} \hat{h}(s) \bigl[ h(s) -\theta_i(s) -\hat{k}(s)\xi_i^{1,s+1} \bigr] - i \sum_{s<t} \sigma(s) \bigl[ \hat{m}(s)\xi_i^{1,s} +\psi_i(s) \bigr] }, \label{eq:Phi} \\
\Omega 
&=& \frac 1N \ln \int \frac{d\vec{u}d\vec{v}}{(2\pi)^{(p-t)t}} e^{i \sum_{\mu>t} \sum_{s<t} u_{\nu,\mu+1}(s) v_{\nu,\mu}(s) } \nonumber \\
& & \times e^{-\frac 12 \sum_{\nu >1,\mu \ge 1} \sum_{s,s'<t} \bigl[ u_{\nu,\mu}(s)Q(s,s')v_{\nu,\mu}(s') + u_{\nu,\mu}(s)K(s,s')v_{\nu,\mu}(s') \bigr] } \nonumber \\
& & \times e^{-\frac 12 \sum_{\nu >1,\mu \ge 1} \sum_{s,s'<t} \bigl[ v_{\nu,\mu}(s)K(s,s')u_{\nu,\mu}(s') + v_{\nu,\mu}(s)q(s,s')v_{\nu,\mu}(s') \bigr] }, \label{eq:Omega}
\end{eqnarray}
with the shorthand $\{ d\vec{h} d\hat{\vec{h}} \} = \prod_i \frac{dh_i(s)d\hat{h}_i(s)}{2\pi}$. 





%---�폜---\subsection{The saddle-point equations}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the limit $N\to\infty$, the integral (\ref{eq:Zbar}) will be dominated by saddle point of the extensive exponent $\Psi+\Phi+\Omega$. %<----------(\ref{eq:Zbar})���̐ϕ��l�́C$N\to\infty$�̋Ɍ��Ŏw������$\Psi+\Phi+\Omega$�̋ɒl�ɂ���Ďx�z�����D�����ŁC$N\to\infty$�Ƃ��ĈƓ_�@�ł��̐ϕ��l��]������D
The saddle-point equations which are derived by differentiation with respect to integration variables $\{ \vec{m},\hat{\vec{m}},\vec{k},\hat{\vec{k}},\vec{q},\hat{\vec{q}},\vec{Q},\hat{\vec{Q}},\vec{K},\hat{\vec{K}} \}$ are as follows: %<----------�܂��C�Ɠ_�������𓱏o����D�ϕ��ϐ�$\{ \vec{m},\hat{\vec{m}},\vec{k},\hat{\vec{k}},\vec{q},\hat{\vec{q}},\vec{Q},\hat{\vec{Q}},\vec{K},\hat{\vec{K}} \}$�ɂ��Ďw��������������ĈƓ_�����߂�ƁC$s,s'<t$�ɑ΂��Ď��̂悤�ɂȂ�D
\begin{eqnarray}
\hat{m}(s)&=&k(s)=0, \\
Q(s,s')&=&\hat{q}(s,s')=0, \\
m(s)&=&\hat{k}(s)=\lim_{N\to\infty}\frac 1N\sum_i<\sigma(s) \xi_i^{1,s}>_i, \label{eq:sp_m} \\
q(s,s')&=&C(s,s')=\lim_{N\to\infty}\frac 1N\sum_i <\sigma(s)\sigma(s')>_i, \label{eq:sp_C} \\
K(s,s')&=&iG(s,s')= i \lim_{N\to\infty} \frac 1N \sum_i \frac{\partial <\sigma(s)>_i}{\partial \theta_i(s')}, \label{eq:sp_G} \\
\hat{Q}(s,s')&=&\biggl. i \lim_{\vec{Q}\to\vec{0}} \frac{\partial \Omega}{\partial K(s,s')} \biggr|_{\rm saddle}, \label{eq:sp_Qhat} \\
\hat{K}(s,s')&=&\biggl. \frac{\partial \Omega}{\partial K(s,s')} \biggr|_{\rm saddle}, \label{eq:sp_Khat} \\
\end{eqnarray}
where $f|_{\rm saddle}$ denotes an evaluation of a function $f$ at the dominating saddle-point, $<\cdot>_i$ denotes 
\begin{eqnarray}
<f(\vec{\sigma},\vec{h},\hat{\vec{h}})>_i
&=& \biggl< \frac{\Tr{\vec{\sigma}} \int \{ d\vec{h} d\hat{\vec{h}} \} W_i(\vec{\sigma},\vec{h},\hat{\vec{h}})f(\vec{\sigma},\vec{h},\hat{\vec{h}})}
        {\Tr{\vec{\sigma}} \int \{ d\vec{h} d\hat{\vec{h}} \} W_i(\vec{\sigma},\vec{h},\hat{\vec{h}})} \biggr>_{\vec{\xi}}, \\
W_i(\vec{\sigma},\vec{h},\hat{\vec{h}})
&=& p_i(\sigma(0)) \biggl[ e^{\sum_{s<t} \bigl( \beta\sigma(s+1)h(s)-\ln 2\cosh \beta h(s) \bigr)  } \nonumber \\
& & \times e^{ i\sum_{s<t} \bigl( \hat{h}(s) \{ h(s)-\theta_i(s)-\hat{k}(s)\xi_i^{1,s+1} \} - \theta(s)\hat{m}(s)\xi_i^{1,s} \bigr)  } \nonumber \\
& & \times e^{-i\sum_{s,s'<t} \bigl( \hat{q}(s,s')\sigma(s)\sigma(s')+\hat{Q}(s,s')\hat{h}(s)\hat{h}(s')+\hat{K}(s,s')\sigma(s)\hat{h}(s') \bigr) } \biggr], 
\end{eqnarray}
and $<\cdot>_{\vec{\xi}}$ denotes the average over the condensed patterns. 
We now calculate the right-hand sides of (\ref{eq:sp_Qhat}) and (\ref{eq:sp_Khat}). 
The eigenvalues $s_\mu$ of a matrix $\vec{S}$ are given by $s_\mu =e^{2\pi i\mu /l}$ (the multiplicity is $p/l$) from $|\lambda \vec{1} - \vec{S}| = |\lambda \vec{1} - \vec{S}'|^{p/l}$. 
Since the matrix $\vec{S}'$ is a unitary matrix, i.e., $\vec{S}'^\dagger\vec{S}'=\vec{1}$, the matrix $\vec{S}$ is also unitary, $\vec{S}^\dagger\vec{S}= {\rm diag}(\vec{S}'^\dagger,\cdots,\vec{S}'^\dagger){\rm diag}(\vec{S}',\cdots,\vec{S}') = {\rm diag}(\vec{S}'^\dagger\vec{S}',\cdots,\vec{S}'^\dagger\vec{S}') = \vec{1}$. 
A $(\mu,\mu)$-element of $(\vec{S^\dagger})^m\vec{S}^n$ becomes $[(\vec{S^\dagger})^m\vec{S}^n]_{\mu\mu} =\delta_{mn}$ by using the unitarity of $\vec{S}$, where $\delta_{mn}$ denotes Kronecker's delta function. 
The identity $(\vec{S})^l=(\vec{S}^\dagger)^l=\vec{S}$ is established because $(\vec{S}')^l=(\vec{S}'^\dagger)^l=\vec{S}'$. 
Therefore, the identity $[(\vec{S^\dagger})^m\vec{S}^n]_{\mu\mu}=\delta_{mn}$ holds for the following value $(m,n)$: 
\begin{equation}
\left\{
\begin{array} {l}
m=m'l+a, \\
n=n'l+a,
\end{array} \right. 
\end{equation}
with $m',n' \in \{0,1,\cdots \}$ and $a \in \{ 0,\cdots,l-1 \}$. 
The multiplicity of the eigenvalues of the matrix $\vec{S}'$ is 1, so ${\rm rank}(\vec{S}'-s_\mu\vec{1})=l-1$. 
Hence, ${\rm rank}(\vec{S}-s_\mu \vec{1})= (p/l) {\rm rank}(\vec{S}'-s_\mu \vec{1}) = p-p/l$. 
%---DELETE---This means that the dimension of the eigenspace with the eigenvalue $s_\mu$ is equal to the multiplicity of eigenvalue of $s_\mu$ for any $\mu$. 
%�Ƃ����悤�ɁC�C�ӂ�$\mu$�ɂ��ČŗL�l$s_\mu$�ɑ΂���$\vec{S}$�̌ŗL��Ԃ̎������ŗL�l$s_\mu$�̏d���x�Ɉ�v����̂ŁC�K���Ȑ����s��$\vec{P}$�ɂ����$\vec{S}$�͎��̂悤�ɑΊp���\�ł���D
%
This means that the matrix $\vec{S}$ can be diagonalized by an appropriate non-singular matrix as 
%�ƂȂ�̂ŁC�K���Ȑ����s��$\vec{P}$�ɂ����$\vec{S}$�͎��̂悤�ɑΊp���\�ł���D%(�C�ӂ�$\mu$�ɂ��ČŗL�l$s_\mu$�ɑ΂���$\vec{S}$�̌ŗL��Ԃ̎������ŗL�l$s_\mu$�̏d���x�Ɉ�v����)%<----------�ƂȂ�̂ŁC�K���Ȑ����s��$\vec{P}$�ɂ����$\vec{S}$�͎��̂悤�ɑΊp���\�ł���D
%
\[
{\rm diag} (\overbrace{s_0,\cdots,s_0}^{p/l},\cdots,\overbrace{s_{l-1},\cdots,s_{l-1}}^{p/l}). 
\]
By working out the saddle-point equation (\ref{eq:sp_Qhat}), $\hat{\vec{Q}}$ becomes as follows: 
\begin{equation}
\hat{Q}(s,s')=-\frac 12 \alpha i\sum_{m,n\ge 0}\lim_{p\to\infty}\sum_{\mu\le p}\{ (\vec{S}\otimes\vec{G})^m[\vec{1}\otimes\vec{C}](\vec{S}^\dagger\otimes\vec{G}^\dagger)^n \}_{\mu\mu}(s',s). 
\end{equation}
Hence $\hat{\vec{Q}}$ is given by 
\begin{equation}
\hat{\vec{Q}} = -\frac 12 \alpha i \sum_{a=0}^{l-1} \sum_{m',n'\ge 0}\vec{G}^{m'l+a} \vec{C}(\vec{G}^\dagger)^{n'l+a}. 
\end{equation}
We define a matrix $\vec{\Gamma}=\vec{S}\otimes\vec{R}$ as having matrix elements $\Gamma_{\mu\mu '}(s,s')=S_{\mu\mu '}R(s,s')$ for $\mu,\mu' \in \{1,\cdots,p \}$ and $s,s' \in \{ 0,\cdots,t-1 \}$ where $\vec{y}=\vec{\Gamma\vec{x}}$ will operate as $y_\mu(s)=\sum_{\mu '>t}\sum_{s'<t}S_{\mu\mu '}R(s,s')x_{\mu '}(s')$ for each $(\mu,s)$. 
Equation (\ref{eq:sp_Khat}) reduces to 
\begin{eqnarray}
\hat{K}(s,s')
&=& -\frac 12 \alpha \frac{\partial}{\partial G(s,s')} \lim_{p\to\infty} \frac 1p \{ \ln\det [\vec{1}\otimes\vec{1}-\vec{S}^\dagger\otimes\vec{G}^\dagger ] + \ln\det [\vec{1}\otimes\vec{1}-\vec{S}\otimes\vec{G}] \} \nonumber \\
&=& -\alpha \frac{\partial}{\partial G(s,s')} \lim_{p\to\infty} \frac 1p \ln \prod_{\mu=0}^{l-1} \biggl( \det [\vec{1}-e^{ 2\pi i \mu /l}\vec{G}^\dagger ] \biggr)^{p/l} \\
\hat{\vec{K}}
&=&\frac{\alpha}l \sum_{\mu=0}^{l-1} e^{2\pi i\mu /l} [\vec{1}-e^{2\pi i\mu /l}\vec{G}^\dagger]^{-1}. 
\end{eqnarray}
Replacing $\xi_i^{1,s}\to\xi^{s}$, we obtain the order parameters of (\ref{eq:spe_m})-(\ref{eq:spe_G}). 






\section{The periodicity of the Gaussian random field \label{app:0}}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



Using time-translation invariant ansatz, i.e., $C(s,s')=C(s-s')$, $C(s)$ also have periodicity as 
\begin{equation}
C(s+l)=C(s), 
\end{equation}
for any $s$, when spin variables $\sigma(s)$ have periodicity, i.e., $\sigma(s+l)=\sigma(s)$. 
It is confirmed that the vector $\vec{v}$ obeys the Gassian distribution 
with mean $\vec{0}$ and covariance matrix $\vec{R}$ from equation (\ref{eq:E_path}). 
The covariance matrix $\vec{R}$ of the Gaussian random fields $\vec{v}$ is given by (\ref{eq:R}). 
Since the matrix $\vec{R}$ is also symmetric Toeplitz matrix, we can put $R(s,s')=R(s-s')$ where $R(s,s')$ are the elements of the matrix $\vec{R}$. 
When $C(s+l)=C(s)$, we approximately get $R(s+l)=R(s)$. 
The correlation coefficient between cross-talk noise $v(s)$ and $v(s+l)$ becomes 
\[
{\rm Corr}(v(s+l),v(s))
=\frac{{\rm Cov}(v(s+l),v(s))}{\sqrt{V(v(s+l))V(v(s))}}
=\frac{R(s+l,s)}{\sqrt{R(s+l,s+l)R(s,s)}}
=1, 
\]
for any $s,s'$. 
The $v(s)$ distribution and the $v(s+l)$ distribution have the same mean and variance, i.e., $E(v(s))=E(v(s+l))=0$ and $V(v(s))=R(s,s)=R(0)=R(s+l,s+l)=V(v(s+l))$. 
Therefore, the identity 
\begin{equation}
v(s+l)=v(s), 
\end{equation}
holds for any $s$. 
Hence, the Gaussian random fields $v(s)$ also have periodicity as $v(s+l)=v(s)$ when spin variables have periodicity as $\sigma(s+l)=\sigma(s)$. 
Therefore, $v(s)$ can be considered as quenched noise. 
After the states $v(0), \cdots , v(l-1)$ occur at random, $v(s)$ continues taking the same values periodically. 
Therefore, we can assume that $v(s)$ is deterministic for a fixed site. 
The Gaussian random fields $v(s)$ are only distributed with respect to site index $i$. 









\section{The Fourier transformation of the symmetric Toeplitz matrix \label{app:1}}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Symmetric Toeplitz matrices can be diagonalized by using the discrete Fourier transformation. 
%�Ώ̃e�v���b�c�s��̓t�[���G�ϊ��ɂ���đΊp���ł���D
%
The Fourier transformation was used to obtain the identity of (\ref{eq:r}). 
%�����ł́C(\ref{eq:r})�������߂邽�߂Ƀt�[���G�ϊ���p���Ă���D
%
The Fourier transformation is defined as 
%�t�[���G�ϊ��������Œ�`����D
%
\begin{equation}
\hat{\xi}_k = \frac 1{\sqrt{t}} \sum_j \xi_j e^{-ikj}, 
\end{equation}
and the inverse Fourier transformation defined as 
%���ɁC�t�t�[���G�ϊ��������Œ�`����D
%
\begin{equation}
\xi_j = \frac 1{\sqrt{t}} \sum_k \hat{\xi}_k e^{ikj},
\end{equation}
where $k$ denotes wave number and its degree of freedom is $t$. 
%�����ŁC$k$�͔g���������C���̎��R�x��$t$�ł���D
%
Each component of $k$ takes the value $0,\frac{2}{t}\pi,\frac{4}{t}\pi, \cdots,\frac{2(t-1)}{t}\pi$.
%$k$�̊e�v�f��$0,\frac{2}{t}\pi,\frac{4}{t}\pi, \cdots,\frac{2(t-1)}{t}\pi$�̒l���Ƃ�D
%
The following symmetric Toeplitz matrix can be diagonalized by using the Fourier representation: 
%���̑Ώ̃e�v���b�c�s��$\vec{D}$�C
%
\begin{equation}
\vec{D}=
\left(
\begin{array}{cccc}
D_0     & D_1     & \cdots & D_{t-1} \\
D_1     & D_0     & \cdots & D_{t-2} \\
\vdots  & \vdots  & \ddots & \vdots  \\
D_{t-1} & D_{t-2} & \cdots & D_0     \\
\end{array}
\right) . 
\end{equation}
The Fourier representation of the quadratic form $\vec{\xi}^T \vec{D} \vec{\xi}$ becomes 
%��2���`��$\vec{\xi}^T \vec{D} \vec{\xi}$�̃t�[���G�\�L�����߂�D
%
\begin{eqnarray}
\vec{\xi}^T \vec{D} \vec{\xi}
&=& \sum_{i=1}^t \sum_{j=1}^t \xi_i D_{|i-j|} \xi_j \nonumber \\
&=& \sum_{\tau=0}^{t-1} \sum_{j=1}^t \xi_j D_\tau \xi_{j-\tau} \nonumber \\
&=& \sum_{\tau} \sum_j \biggl( \frac 1{\sqrt{t}} \sum_{k_1} \hat{\xi}_{k_1} e^{ik_1j} \biggr)  D_\tau \biggl( \frac 1{\sqrt{t}} \sum_{k_2} \hat{\xi}_{k_2} e^{ik_2(j-\tau)} \biggr) \nonumber \\
&=& \sum_k \hat{\xi}_k \biggl( \sum_{\tau} D_\tau e^{ik\tau} \biggr) \hat{\xi}_{-k}, 
\end{eqnarray}
where the index $j$ of the variables $\xi_j$ is understood to be taken modulo $t$. 
%�����ŁC$\xi_j$�̓Y������$ \mod t$���Ƃ�Ƃ���D
%
Therefore, the Fourier transformation of the symmetric Toeplitz matrix $\vec{D}=(D_\tau)$ is given by 
%����āC�Ώ̃e�v���b�c�s��$\vec{D}=(D_\tau)$�̃t�[���G�ϊ�$\hat{\vec{D}}=(\hat{D}_k)$�́C
%
\begin{equation}
\hat{D}_k=\sum_{\tau} D_\tau e^{ik\tau}. 
\end{equation}
%�ƂȂ�D
This transformation is also called the lattice Green's function. 
%�܂��C���̊֐��͊i�q�O���[���֐��Ƃ��Ă΂��D
%










\section{The signal-to-noise analysis of the finite-step sequence processing model \label{app:2}}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We discuss the stability of limit cycles by means of the signal-to-noise analysis. 
Let us consider the following deterministic synchronous dynamics: 
\begin{equation}
x_i^{t+1} = F ( \sum_{j=1}^N J_{ij} x_j^t ) , \label{eq:eq-st}
\end{equation}
where the $x_i^t$ represents the state of the $i$-th neuron at time $t$ and $F(\cdot)$ denotes an output function. 
The retrieval state converges to some limit cycle with $l$ steps. 
We introduced the Poincar\'{e} map to get the states every $l$ steps in the steady state. 
The periodic state can be transformed into a stable state by using this map. 
Hence, we can discuss the properties of a stability of limit cycles. 

Let us consider the case of convergence to periodic states of the limit cycle retrieval. 
We assume $x_i^t = x_i^{t-l}$ in (\ref{eq:eq-st}). 
The overlap between the $\mu$-th memory pattern $\vec{\xi}^{\nu \mu}$ of the $\nu$-th limit cycle and the network state $\vec{x}$ is defined as 
\begin{equation}
    m_{\nu \mu}^t = \frac 1N \sum_{i=1}^N \xi_i^{\nu \mu} x_i^t. 
    \label{eq:overlap-t}
\end{equation}
The dynamics (\ref{eq:eq-st}) can be rewritten as 
\begin{eqnarray}
    x_i^{t+1} &=& F(h_i^t) \label{eq:eq-st-t} \\
    h_i^t
    &=& \sum_{j=1}^N J_{ij} x_j^t
     =  \sum_{\nu=1}^{p/l} \sum_\mu \xi_i^{\nu \mu +1} m_{\nu \mu}^t, 
    \label{eq:hi-t1}
\end{eqnarray}
where $h_i^t$ is a local field. 
Now let us consider the case to retrieve the 1st limit cycle $\mbox{\boldmath $\xi$}^{1 \mu}$, $\mu \in \{ 1, \cdots ,l \}$. 
We assume the memorized pattern of another limit cycles $\mbox{\boldmath $\xi$}^{\nu \mu}, \nu \ne 1$ does not have a finite overlap. 
We consider retrieval solutions in which $m_{1 \mu}^t \sim O(1)$, and $m_{\nu \mu}^t \sim O( 1 / \sqrt{N} ), \nu \geq 2 $. 
We assume that the components $x_i$ of the equilibrium state $\vec{x}$ are independent on the unit number $i$ in the limit $N \to \infty$. 
It is necessary to assume that the self-averaging property to holds so that the site average can be replaced by an average over the random patterns and random variable $\vec{x}$. 
In this situation the overlap $m_{\nu\mu}^t$ need not to be a random variable.
\begin{eqnarray}
    m_{\nu \mu}^t &=& \bar{m}_{\nu \mu}^t + U_t m_{\nu \mu}^{t-1} ,
    \label{eq:overlap-t2} \\
    \bar{m}_{\nu \mu}^t &=& \frac 1N \sum_i \xi_i^{\nu ,\mu}
                            x_i^{t (\nu \mu )}, \label{eq:m-bar} \\
    U_t &=& \frac 1N \sum_i x_i^{'t (\nu \mu )},
\end{eqnarray}
where
\begin{eqnarray}
   x_i^{t(\nu \mu )}  &=& F ( \sum_{(\nu ' \mu ') \ne (\nu ,\mu -1)}
                              \xi^{\nu ' \mu '+1} m_{\nu ' ,\mu '}^{t-1} )
\\
   x_i^{'t(\nu \mu )} &=& F'( \sum_{(\nu ' \mu ') \ne (\nu ,\mu -1)}
                              \xi^{\nu ' \mu '+1} m_{\nu ' \mu '}^{t-1} ) .
\end{eqnarray}
By applying (\ref{eq:overlap-t2}) repeatedly, we obtain 
\begin{eqnarray}
    m_{\nu \mu}^t
    &=& \bar{m}_{\nu \mu}^t
      + U_t         \bar{m}_{\nu \mu -1}^{t-1}
      + U_t U_{t-1} \bar{m}_{\nu \mu -2}^{t-2} + \cdots \nonumber \\
    & & \; \; \; \;
      + U_t U_{t-1} \cdots U_{t-l+2} \bar{m}_{\nu \mu -l+1}^{t-l+1}
      + U_t U_{t-1} \cdots U_{t-l+1} m_{\nu \mu -l}^{t-l}. 
\end{eqnarray}
Since the retrieval state is assumed to be steady, i.e., $x_i^{t-l}=x_i^t $, the identity $m_{\nu \mu -l}^{t-l} = m_{\nu \mu}^t$ holds. 
Therefore the overlap becomes 
\begin{equation}
    m_{\nu \mu} ^t
    = (1-\prod_{k=0}^{l-1}U_{t-k})^{-1} \left[
       \bar{m}_{\nu \mu}^t
       + \sum_{k=1}^{l-1}\prod_{k'=0}^{k-1} U_{t-k'} \bar{m}_{\nu
\mu -k}^{t-k}
       \right] .
    \label{eq:overlap-t3}
\end{equation}
Substituting (\ref{eq:m-bar}) into (\ref{eq:overlap-t3}), we obtain 
\begin{eqnarray}
    m_{\nu \mu} ^t
    &=& \frac 1N (1-\prod_{k=0}^{l-1}U_{t-k})^{-1} \left[
        \sum_{j=1}^N \xi_j^{\nu \mu} x_j^{t (\nu \mu )} \right.
        \nonumber \\
    & & \; \; \; \; \;
      + \left.
        \sum_{j=1}^N \xi_j^{\nu ,\mu} x_j^{t (\nu \mu )}
        \sum_{k=1}^{l-1} ( \prod_{k'=0}^{k-1} U_{t-k'} )
        \sum_{j=1}^N \xi_j^{\nu ,\mu -k} x_j^{t-k (\nu \mu -k)}
        \right] ,
    \label{eq:overlap-t4}
\end{eqnarray}
Replacing $\xi_i^{1 \mu} \to \xi_i^\mu$ and $m_{1 \mu}^t \to m_\mu^t $, and substituting (\ref{eq:overlap-t4}) into (\ref{eq:hi-t1}), the local field $h_i^t$ can be rewritten as 
\begin{eqnarray}
    h_i^t
    &=& \sum_\mu \xi_i^{\mu +1} m_\mu^t \nonumber \\
    & & \; \; \; \; \;
      + \frac 1N (1-\prod_{k=0}^{l-1}U_{t-k})^{-1} \left[
        \sum_{\nu \geq 2}^{\alpha N /l} \sum_{\mu =1}^l
        \xi_i^{\nu \mu +1} \xi_i^{\nu \mu } x_i^{t (\nu \mu )} \right.
        \nonumber \\
    & & \; \; \; \; \; \; \; \; \; \;
      + \left.
        \sum_{k=0}^{l-1} ( \prod_{k'=0}^{k-1} U_{t-k'} )
        \sum_{\nu \geq 2}^{\alpha N /l} \sum_{\mu =1}^l
        \xi_i^{\nu \mu +1} \xi_i^{\nu \mu -k} x_i^{t-k (\nu \mu -k)}
        \right]
        \nonumber \\
    & & \; \; \; \; \;
      + \frac 1N (1-\prod_{k=0}^{l-1}U_{t-k})^{-1} \left[
        \sum_{\nu \geq 2}^{\alpha N /l} \sum_{\mu =1}^l \sum_{j \ne i}^N
        \xi_i^{\nu \mu +1} \xi_j^{\nu \mu } x_j^{t (\nu \mu)} \right.
        \nonumber \\
    & & \; \; \; \; \; \; \; \; \; \;
      + \left.
        \sum_{k=0}^{l-1} ( \prod_{k'=0}^{k-1} U_{t-k'} )
        \sum_{\nu \geq 2}^{\alpha N /l} \sum_{\mu =1}^l \sum_{j \ne i}^N
        \xi_i^{\nu \mu +1} \xi_j^{\nu \mu -k} x_j^{t-k (\nu \mu -k)}
        \right] .
    \label{eq:hi-t2}
\end{eqnarray}
The first term in (\ref{eq:hi-t2}) is regarded as the signal. 
The second and the third terms are regarded as the mean and the variance of the crosstalk noise, respectively. 
In the second term, if the suffix is $\mu + 1 \equiv \mu -k \, (\mod l)$, $\xi_i^{\nu \mu +1}$ and $\xi_j^{\nu \mu -k}$ are represented the same patterns.
Since these terms consist of uncorrelated random variables with the order $O(1/\sqrt{N})$, the other terms in the second term can be omitted. 
The term of $k=l-1$ is only one remaining of the terms with $k \in \{ 0, \cdots , l-1 \}$. 
Hence we can estimate the second term as 
\begin{eqnarray}
    & & \frac 1N (\prod_{k'=0}^{l-2} U_{t-k'} )
        (1-\prod_{k'=0}^{l-1}U_{t-k'})^{-1}
        \sum_{\nu \geq 2}^{\alpha N/l} \sum_{\mu =1}^l
        \xi_i^{\nu \mu +1} \xi_j^{\nu \mu -l+1} x_j^{t-l+1(\nu \mu -l+1)}
        \nonumber \\
    &=& \alpha (\prod_{k'=0}^{l-2} U_{t-k'} )
        (1-\prod_{k'=0}^{l-1}U_{t-k'})^{-1} x_i^{t+1} .
    \label{eq:hi-t2-2nd}
\end{eqnarray}
We assume the third term is normally distributed with mean 0 and variance $\sigma_t^2$ because of the independence of $x_i^{t (\nu \mu )}$ and $\xi_i^{\nu \mu}$. 
The variance $\sigma_t^2$ of the cross talk noise is estimated as 
\begin{equation}
    \sigma_t^2
      = \alpha (1-\prod_{k'=0}^{l-1}U_{t-k'})^{-2}
        \left[
        q^t
      + \sum_{k=1}^{l-1}
        (\prod_{k'=0}^{k-1} U_{t-k'})^2
        q^{t-k}
        \right] ,
\end{equation}
where
\begin{equation}
    q^{t-k} = \frac 1N\sum_{j \ne i}^N (x_j^{t-k})^2 .
\end{equation}
The local field $h_i$ is obtained by setting $z_i \sim N(0,1)$ in (\ref{eq:hi-t2}), 
\begin{equation}
    h_i = \sum_{\mu =1}^s \xi_i^{\mu +1} m_\mu^t
        + \alpha (\prod_{k'=0}^{l-2} U_{t-k'} )
          (1-\prod_{k'=0}^{l-1}U_{t-k'})^{-1} x_i^{t+1}
        + \sigma_t z_i .
    \label{eq:hi-t4}
\end{equation}
The self-averaging property is assumed. 
Replacing $x_i \to Y$ and $\xi_i^\mu \to \xi^\mu$, we obtain the macroscopic equations as follows, 
\begin {eqnarray}
    Y^{t+1}(\xi^1, \cdots ,\xi^l;z)
      &=& F(\sum_{\mu =1}^l \xi^{\mu +1} m_\mu^t
                + \Gamma Y^{t+1}(\xi^1, \cdots ,\xi^l;z) + \sigma_t z ) \\
    m_\mu^{t+1} &=& \int^\infty_{-\infty}Dz\;
                \ll \xi^\mu Y^{t+1}(\xi^1, \cdots ,\xi^l;z) \gg  \\
    q^{t+1} &=& \int^\infty_{-\infty}Dz\; \ll
                {Y^{t+1}(\xi^1, \cdots ,\xi^l;z)}^2 \gg \\
    U^{t+1} &=& \frac 1\sigma_t \int^\infty_{-\infty}Dz \;
                z \ll Y^{t+1}(\xi^1, \cdots ,\xi^l;z) \gg \\
    \sigma_{t+1}^2 &=&
                \alpha (1-\prod_{k'=0}^{l-1}U_{t-k'})^{-2}
                \left[ q^t + \sum_{k=1}^{l-1}
                (\prod_{k'=0}^{k-1} U_{t-k'}) q^{t-k} \right] \\
    \Gamma  &=& \alpha (\prod_{k'=0}^{l-2} U_{t-k'} )
                (1-\prod_{k'=0}^{l-1}U_{t-k'})^{-1}
\end {eqnarray}
Now let us consider the case that the state of memory retrieval is periodic. 
We can set 
\begin{equation}
    m_\nu^t=m \delta_{\nu t}, \; \;
    q_t=q, \; \; U_t=U, \; \;
    \sigma_t=\sigma, \; \;
    (t=1, \cdots ,l) ,
\end{equation}
where $\delta_{\nu t}$ denotes Kronecker's delta. 
Finally, we obtain the macroscopic equations as follows: 
\begin {eqnarray}
    Y(\xi^{t+1};z)
      &=& F(\xi^{t+1} m
            + \Gamma Y(\xi^{t+1};z) + \sigma z )
          \label{eq:opeq-limit1} \\
    m &=& \int^\infty_{-\infty}Dz\;
          \ll \xi^{t+1} Y(\xi^{t+1};z) \gg \\
    q &=& \int^\infty_{-\infty}Dz\;
          \ll {Y(\xi^{t+1};z)}^2 \gg \\
    U &=& \frac 1\sigma \int^\infty_{-\infty}Dz \;
          z \ll Y(\xi^{t+1};z) \gg \\
    \sigma^2 &=& \alpha \frac {1-U^{2l}} {(1-U^l)^2(1-U^2)} q \\
    \Gamma &=& \frac {\alpha U^{l-1}} {1-U^l} .
          \label{eq:opeq-limit2}
\end {eqnarray}
Setting $F( \cdot )= \sgn ( \cdot )$ and using Maxwell rule \cite{shiino1992}, we obtain (\ref{eq:sna_m})-(\ref{eq:sna_rho}) as follows: 
\begin{eqnarray}
m &=& {\rm erf} \biggl( \frac m{\sqrt{2\alpha \rho}} \biggr), \\
U &=& \sqrt{\frac 2{\pi\alpha \rho}}e^{-\frac {m^2}{2\alpha \rho}}, \\
\rho &=& \frac{1-U^{2l}}{(1-U^2)(1-U^l)^2}, 
\end{eqnarray}
where $\sigma^2=\alpha \rho q$, $q=1$ and $\sgn (\cdot)$ denotes the sign function ($\sgn (x)=1$ for $x \ge 0$, $-1$ for $x<0$). 
Thus, we find that these stationary state equations of the order parameters given by the signal-to-noise analysis (SCSNA) 
are equivalent to those of the path-integral analysis (\ref{eq:sna_m})-(\ref{eq:sna_rho}). 







\begin{thebibliography}{99}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


\bibitem{hopfield1982}
Hopfield J J 1982 
Neural networks and physicalsystem with emergent collective computational abilities 
{\it Proc. Natl. Acad. Sci. USA} {\bf 79} 2554. 

\bibitem{shiino1992}
Shiino M and Fukai T 1992 
Self-consistent signal-to-noise analysis and its application to analogue neural networks with asymmetric connections 
{\it J. Phys. A: Math. Gen.} {\bf 25} L375. 

\bibitem{amit1985b}
Amit D J, Gutfreund H and Sompolinsky, H 1985 
Storing infinite numbers of patterns in a spin-glass model of neural networks 
{\it Physical Review Letters} {\bf 55} 1530.

\bibitem{kuhn1991}
K\"{u}hn S, B\"{o}s S and van Hemmen J L 1991 
Statistical mechanics for networks of graded-response neurons 
{\it Physical Review A} {\bf 43} 2084.

\bibitem{mimura1995}
Mimura K and Okada M 1995 
The S/N analysis of associative memory: sign alternating and paring memorization method 
{\it JNNS Transactions} {\bf 2} 38. %(in Japanese).

\bibitem{okada1996}
Okada M 1996 
Notions of associative memory and sparse coding 
{\it Neural Networks} {\bf 9} 1429. 

\bibitem{mimura1996}
Mimura K, Okada M and Kurata K 1996 
Capacity of autocorrelation associative memory with quantized synaptic weight 
{\it Systems \& Computers in Japan} {\bf 27} 55. 

\bibitem{mimura1998a}
Mimura K, Okada M and Kurata K 1998 
Robustness to noise of associative memory using nonmonotonic analogue neurons 
{\it IEICE Transaction on Information and systems} {\bf E81-D} 928. 

\bibitem{mimura1998b}
Mimura K, Okada M and Kurata K 1998 
Associative memory model with forgetting process using nonmonotonic neurons 
{\it IEICE Transaction on Information and systems} {\bf E81-D} 1298.

\bibitem{mimura2003}
Mimura K, Kimoto T and Okada M 2003 
Synapse efficiency diverges due to synaptic pruning following over-growth 
{\it Physical Review E} {\bf 68} 031910. 

\bibitem{okada1995}
Okada M 1995 
A hierarchy of macrodynamical equations for associative memory 
{\it Neural Networks} {\bf 8} 833. 

\bibitem{during1998}
D${\rm \ddot{u}}$ring A, Coolen A C C and Sherrington D 1998 
Phase diagram and storage capacity of sequence processing neural networks 
{\it J. Phys. A: Math. Gen.} {\bf 31} 8609. 

\bibitem{kawamura2002}
Kawamura M and Okada M 2002 
Transient dynamics for sequence processing neural networks 
{\it J. Phys. A: Math. Gen.} {\bf 35} 253. 

\bibitem{coolen1993}
Coolen A C C and Sherrington D 1993 
Dynamics of fully connected attractor neural networks near saturation 
{\it Physical Review Letters} {\bf 71} 3886. 

\bibitem{coolen2000}
Coolen A C C 2000 
Statistical mechanics of recurrent neural networks II. Dynamics 
{\it Preprint} cont-mat/0006011 .

\bibitem{amari1988a}
Amari S and Maginu K 1988 
Statistical neurodynamics of associative memory 
{\it Neural Networks} {\bf 1} 63. 

\bibitem{amari1988b}
Amari S 1988 
Statistical neurodynamics of various versions of correlation associative memory 
{\it Proceedings of IEEE Conference on Neural Networks} {\bf 1} 633. 

\bibitem{gardner1987}
Gardner E, Derrida B and Mottishaw P 1987 
Zero temperature parallel dynamics for infinite range spin glasses and neural networks 
{\it Journal de Physique} {\bf 48} 741.

\bibitem{bolle2003}
Bolle D, Blanco J B and Verbeiren T 2003 
The signal-to-noise analysis of the Little-Hopfield model revisited 
{\it Preprint} cond-mat/0307499. 



%\bibitem{coolen2001}
%Coolen A C C, Heimel A and Sherrington D 2001 
%Dynamics of the batch minority game with inhomogeneous decision noise 
%{\it Physical Review E}, {\bf 65}, 016126. 

%\bibitem{inoue2001}
%Inoue J and Coolen A C C 2001 
%Dynamics of on-line Hebbian learning with structurally unrealizable restricted training sets 
%{\it J. Phy. A: Math. Gen.} {\bf 34} L401. 



%\bibitem{sourlas1989}
%Sourlas N 1989 
%Spin-glass models as error-correcting codes 
%{\it Nature} {\bf 339} 693.

%\bibitem{sommers1987}
%Sommers H J 1987 
%Path-integral approach to Ising spin-grass dynamics 
%{\it Phys. Rev. Lett.} {\bf 71} 3886. 

%\bibitem{abeles1983}
%Abeles M 1983 
%The quantification and graphic display of correlations among three spike trains.
%{\it IEEE Trans. BME} {\bf 30} 235. 

%\bibitem{abeles1993}
%Abeles M, Prut Y, Bergman H, Vaadia E and Aertsen A 1993 
%Integration. synchronicity, and periodicity, 
%in {\it Spatio-temporal Aspects of Brain Function} (edited by Aertsen A and von Seelen W (Elsever, Amsterdam)).

%\bibitem{hermann1995}
%Hermann M, Hertz J A and Pr\"{u}gel-Bennet A 1995 
%Analysis of syn-fire chains 
%{\it Network: Computation in Neural Systems} {\bf 6} 403. 

\end{thebibliography}

\end{document}







cond-mat0310268.extracted_bib
\begin{thebibliography}{99}
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


\bibitem{hopfield1982}
Hopfield J J 1982 
Neural networks and physicalsystem with emergent collective computational abilities 
{\it Proc. Natl. Acad. Sci. USA} {\bf 79} 2554. 

\bibitem{shiino1992}
Shiino M and Fukai T 1992 
Self-consistent signal-to-noise analysis and its application to analogue neural networks with asymmetric connections 
{\it J. Phys. A: Math. Gen.} {\bf 25} L375. 

\bibitem{amit1985b}
Amit D J, Gutfreund H and Sompolinsky, H 1985 
Storing infinite numbers of patterns in a spin-glass model of neural networks 
{\it Physical Review Letters} {\bf 55} 1530.

\bibitem{kuhn1991}
K\"{u}hn S, B\"{o}s S and van Hemmen J L 1991 
Statistical mechanics for networks of graded-response neurons 
{\it Physical Review A} {\bf 43} 2084.

\bibitem{mimura1995}
Mimura K and Okada M 1995 
The S/N analysis of associative memory: sign alternating and paring memorization method 
{\it JNNS Transactions} {\bf 2} 38. %(in Japanese).

\bibitem{okada1996}
Okada M 1996 
Notions of associative memory and sparse coding 
{\it Neural Networks} {\bf 9} 1429. 

\bibitem{mimura1996}
Mimura K, Okada M and Kurata K 1996 
Capacity of autocorrelation associative memory with quantized synaptic weight 
{\it Systems \& Computers in Japan} {\bf 27} 55. 

\bibitem{mimura1998a}
Mimura K, Okada M and Kurata K 1998 
Robustness to noise of associative memory using nonmonotonic analogue neurons 
{\it IEICE Transaction on Information and systems} {\bf E81-D} 928. 

\bibitem{mimura1998b}
Mimura K, Okada M and Kurata K 1998 
Associative memory model with forgetting process using nonmonotonic neurons 
{\it IEICE Transaction on Information and systems} {\bf E81-D} 1298.

\bibitem{mimura2003}
Mimura K, Kimoto T and Okada M 2003 
Synapse efficiency diverges due to synaptic pruning following over-growth 
{\it Physical Review E} {\bf 68} 031910. 

\bibitem{okada1995}
Okada M 1995 
A hierarchy of macrodynamical equations for associative memory 
{\it Neural Networks} {\bf 8} 833. 

\bibitem{during1998}
D${\rm \ddot{u}}$ring A, Coolen A C C and Sherrington D 1998 
Phase diagram and storage capacity of sequence processing neural networks 
{\it J. Phys. A: Math. Gen.} {\bf 31} 8609. 

\bibitem{kawamura2002}
Kawamura M and Okada M 2002 
Transient dynamics for sequence processing neural networks 
{\it J. Phys. A: Math. Gen.} {\bf 35} 253. 

\bibitem{coolen1993}
Coolen A C C and Sherrington D 1993 
Dynamics of fully connected attractor neural networks near saturation 
{\it Physical Review Letters} {\bf 71} 3886. 

\bibitem{coolen2000}
Coolen A C C 2000 
Statistical mechanics of recurrent neural networks II. Dynamics 
{\it Preprint} cont-mat/0006011 .

\bibitem{amari1988a}
Amari S and Maginu K 1988 
Statistical neurodynamics of associative memory 
{\it Neural Networks} {\bf 1} 63. 

\bibitem{amari1988b}
Amari S 1988 
Statistical neurodynamics of various versions of correlation associative memory 
{\it Proceedings of IEEE Conference on Neural Networks} {\bf 1} 633. 

\bibitem{gardner1987}
Gardner E, Derrida B and Mottishaw P 1987 
Zero temperature parallel dynamics for infinite range spin glasses and neural networks 
{\it Journal de Physique} {\bf 48} 741.

\bibitem{bolle2003}
Bolle D, Blanco J B and Verbeiren T 2003 
The signal-to-noise analysis of the Little-Hopfield model revisited 
{\it Preprint} cond-mat/0307499. 



%\bibitem{coolen2001}
%Coolen A C C, Heimel A and Sherrington D 2001 
%Dynamics of the batch minority game with inhomogeneous decision noise 
%{\it Physical Review E}, {\bf 65}, 016126. 

%\bibitem{inoue2001}
%Inoue J and Coolen A C C 2001 
%Dynamics of on-line Hebbian learning with structurally unrealizable restricted training sets 
%{\it J. Phy. A: Math. Gen.} {\bf 34} L401. 



%\bibitem{sourlas1989}
%Sourlas N 1989 
%Spin-glass models as error-correcting codes 
%{\it Nature} {\bf 339} 693.

%\bibitem{sommers1987}
%Sommers H J 1987 
%Path-integral approach to Ising spin-grass dynamics 
%{\it Phys. Rev. Lett.} {\bf 71} 3886. 

%\bibitem{abeles1983}
%Abeles M 1983 
%The quantification and graphic display of correlations among three spike trains.
%{\it IEEE Trans. BME} {\bf 30} 235. 

%\bibitem{abeles1993}
%Abeles M, Prut Y, Bergman H, Vaadia E and Aertsen A 1993 
%Integration. synchronicity, and periodicity, 
%in {\it Spatio-temporal Aspects of Brain Function} (edited by Aertsen A and von Seelen W (Elsever, Amsterdam)).

%\bibitem{hermann1995}
%Hermann M, Hertz J A and Pr\"{u}gel-Bennet A 1995 
%Analysis of syn-fire chains 
%{\it Network: Computation in Neural Systems} {\bf 6} 403. 

\end{thebibliography}