The Parisi formula for mixed $p$-spin models.
The Parisi formula for the free energy in the Sherrington-Kirkpatrick and mixed $p$-spin models for even $p\geq 2$ was proved in the seminal work of Michel Talagrand, \cite{T-P}. In this paper we prove the Parisi formula for general mixed $p$-spin models which also include $p$-spin interactions for odd $p$. Most of the ideas used in the paper are well known and can now be combined following a recent proof of the Parisi ultrametricity conjecture in \cite{PU}.
Parisi.tex
 \documentclass[12pt]{article}
%\usepackage{amsmath}
%\usepackage{amssymb}
\usepackage{amsmath, amsthm, amssymb}
\usepackage[dvips]{graphicx}
\usepackage{epsfig}
%\usepackage{subfigure}
\usepackage{vector}
\textwidth=6.5in
\textheight=8.9in
\topmargin=-0.2in
\oddsidemargin=0.2in
\evensidemargin=0.3in
\def\eps{\varepsilon}
\parindent=8mm
\frenchspacing
\font\tencmmib=cmmib10 \skewchar\tencmmib '60
\newfam\cmmibfam
\textfont\cmmibfam=\tencmmib
\def\Ph{\mathchar'4010}
\def\Ps{\mathchar'4011}
\font\tensmc=cmcsc10
\def\smc{\tensmc}
\def\srm{\sevenrm }
\font\tenmsb=msbm10
\font\eightmsb=msbm10 scaled 1100
\def\Bbb#1{\hbox{\tenmsb#1}}
\def\Bbbb#1{\hbox{\eightmsb#1}}
\def\bbox{\quad\hbox{\vrule \vbox{\hrule \vskip2pt \hbox{\hskip2pt
\vbox{\hsize=1pt}\hskip2pt} \vskip2pt\hrule}\vrule}}
\def\lessim{\ \lower4pt\hbox{$
\buildrel{\displaystyle <}\over\sim$}\ }
\def\gessim{\ \lower4pt\hbox{$\buildrel{\displaystyle >}
\over\sim$}\ }
\def\n{\noindent}
\def\P{{\cal P}}
\def\si{\bar{\sigma}}
\def\E{{\cal E}}
\def\LL{{\cal L}}
\def\FF{{\cal F}}
\def\SS{{\cal S}}
\def\C{{\cal C}}
\def\H{{\cal H}}
\def\X{{\cal X}}
\def\Y{{\cal Y}}
\def\A{{\cal A}}
\def\O{{\cal O}}
\def\M{{\cal M}}
\def\G{{\Gamma}}
\def\eps{{\varepsilon}}
\def\ch{{\mbox{\rm ch}\hspace{0.4mm}}}
\def\sh{{\mbox{sh}}}
\def\th{{\mbox{th}}}
\def\Av{{\mbox{\rm{Av}}}}
\def\n{\noindent}
\def\bbe{\vskip .15pc\hangindent=.5pc\hangafter=1}
\def\la{{\Bigl\langle}}
\def\ra{{\Bigr\rangle}}
\def\goin{\to\infty}
\def\qed{\hfill\break\rightline{$\bbox$}}
%\magnification=\magstep1
\parskip=0.03truein
\hfuzz=0.3truein

\newcommand{\e}{\mathbb{E}}
\newcommand{\p}{\mathbb{P}}
\newcommand{\Reals}{\mathbb{R}}
\newcommand{\Natural}{\mathbb{N}}
\newcommand{\calA}{\mathcal{A}}
\newcommand{\F}{{\cal F}}
\newcommand{\vsi}{{\vec{\sigma}}}
\newcommand{\vrho}{{\vec{\rho}}}
\newcommand{\Exval}[1]{\mathbb{E}#1}
\newcommand{\prob}[1]{\mathbb{P}(#1)}
\newcommand{\Prob}[1]{\mathbb{P}\{#1\}}
\newcommand{\bigProb}[1]{\mathbb{P}\Bigl\{#1\Bigr\}}
\newcommand{\bigprob}[1]{\mathbb{P}\left(#1\right)}


\newtheorem{proposition}{Proposition}
\newtheorem{lemma}{Lemma}
\newtheorem{theorem}{Theorem}
\newtheorem{corollary}{Corollary}
%\newtheorem*{theorem1prime}{Theorem 1$^\prime$}


\makeatletter
\@addtoreset{equation}{section}
\renewcommand{\theequation}{\thesection.\arabic{equation}}
\makeatother
\input invlat.tex

\begin{document}

\title{The Parisi formula for mixed $p$-spin models.}
\author{Dmitry Panchenko\thanks{Department of Mathematics, Texas A\&M University, email: panchenk@math.tamu.edu. 
Partially supported by NSF grant.}\\
%{\it Texas A\&M University}
}
%\date{}
\maketitle
\begin{abstract} 
The Parisi formula for the free energy in the Sherrington-Kirkpatrick and mixed $p$-spin models for even $p\geq 2$ was proved in the seminal work of Michel Talagrand, \cite{T-P}. In this paper we prove the Parisi formula for general mixed $p$-spin models which also include $p$-spin interactions for odd $p$. Most of the ideas used in the paper are well known and can now be combined following a recent proof of the Parisi ultrametricity conjecture in \cite{PU}.

\end{abstract}
\vspace{0.5cm}
Key words: Sherrington-Kirkpatrick model, free energy, ultrametricity.

\section{Introduction and main result.}
The formula for the free energy in the Sherrington-Kirkpatrick model \cite{SK} was famously discovered by G. Parisi in  \cite{Parisi79},  \cite{Parisi} using the approach that combined a replica trick with a very special choice of the replica matrix. It was later understood in \cite{M1}, \cite{M2} that the special form of the replica matrix conjectured by Parisi corresponded to a number of physical properties of the Gibbs measure of the model, one of them being the ultrametricity of its support. The Parisi formula for the free energy in the Sherrington-Kirkpatrick and mixed $p$-spin models was proved by M. Talagrand in \cite{T-P} following the discovery of the replica symmetry breaking interpolation scheme by F. Guerra in \cite{Guerra}. However, for technical reasons only the case of $p$-spin interactions for even $p\geq 2$ was considered.  Using the main result in \cite{PU} which yields that under a small perturbation of the Hamiltonian the support of the Gibbs measure in these models is indeed asymptotically ultrametric, we prove the Parisi formula for general mixed $p$-spin models that include odd $p$-spin interactions as well.  Let $N\geq 1$. Let us consider Gaussian processes $H_{N,p}(\vsi)$ for $p\geq 1$ indexed by $\vsi\in\Sigma_N = \{-1,+1\}^N$, called pure $p$-spin Hamiltonians,
\begin{equation}
H_{N,p}(\vsi)
=
\frac{1}{N^{(p-1)/2}}
\sum_{1\leq i_1,\ldots,i_p\leq N}g_{i_1,\ldots,i_p} \sigma_{i_1}\ldots\sigma_{i_p},
\label{HamSKp}
\end{equation}
where random variables $(g_{i_1,\ldots,i_p})$ are standard Gaussian independent for all $p\geq 1$ and all $(i_1,\ldots,i_p)$. Let us define a mixed $p$-spin Hamiltonian as their linear combination 
\begin{equation}
H_N(\vsi) = \sum_{p\geq 1} \beta_p H_{N,p}(\vsi)
\label{HN}
\end{equation}
with coefficients $(\beta_p)$ that decrease fast enough, for example,  $\sum_{p\geq 1} 2^p \beta_p^2<\infty$. This technical condition is sufficient to ensure that the process is well defined when the sum includes infinitely many terms. The covariance of the Gaussian process $H_N(\vsi)$ is easy to compute and is given by a function of the normalized scalar product, called overlap, $R_{1,2}= N^{-1}\sum_{i\leq N} \sigma^1_i \sigma_i^{2}$ of spin configurations $\vsi^1$ and  $\vsi^2$,
\begin{equation}
 \e H_N(\vsi^1) H_N(\vsi^2) = N\xi(R_{1,2}), 
 \end{equation}
where $\xi(x)=\sum_{p\geq 1}\beta_p^2 x^p$.  Given $k\geq 1,$ let us consider two sequences of parameters,
\begin{equation}
0=m_0\leq m_1\leq \ldots \leq m_{k-1}\leq m_k\leq 1
\label{m}
\end{equation} 
and
\begin{equation}
0=q_0\leq q_1\leq \ldots \leq q_{k}\leq q_{k+1} = 1,
\label{q}
\end{equation}
which will be denoted by $\vec{m}$ and $\vec{q}$ and  consider independent Gaussian random variables  $(z_j)_{0\leq j\leq k}$ with variances $\e z_j^2 = \xi'(q_{j+1}) - \xi'(q_j).$
We define 
\begin{equation}
X_{k+1}=\log \ch \sum_{0\leq j\leq k} z_j 
\,\,\mbox{ and }\,\,
X_l=\frac{1}{m_l}\log \e_l\exp m_l X_{l+1}
\label{Xl}
\end{equation}
recursively for $l\leq k,$ where $\e_l$ denotes the expectation in the r.v. $(z_j)_{j\geq l}.$
When $m_l=0$ this means $X_l=\e_l X_{l+1}.$  Let us denote $\theta(q)=q\xi'(q)-\xi(q)$ and define
\begin{equation}
\P_k(\vec{m},\vec{q})
=
 X_0(\vec{m},\vec{q})
-\frac{1}{2}
\sum_{1\leq j\leq k} m_j \bigl(\theta(q_{j+1}) - \theta(q_j)\bigr).
\label{Pk}
\end{equation}
Then the following theorem holds.
\begin{theorem}\label{Th1} (The Parisi formula) We have,
\begin{equation}
\lim_{N\to\infty} 
\frac{1}{N} \, \e \log \sum_{\vsi\in\Sigma_N} \exp H_N(\vsi)
= 
\inf \P_k(\vec{m},\vec{q}),
\label{Parisi}
\end{equation}
where the infimum is taken over all $k, \vec{m}$ and $\vec{q}$ as above. 
\end{theorem}\noindent
The quantity in the limit on the left hand side is called the free energy of the model and the infimum on the right hand side is the famous Parisi formula. One can include the external field term in the model but for simplicity of notation we will omit it. 



\section{Proof.}

Most of the ideas of the proof are well known and available in different places in the literature. Under various formulations of the ultrametricity conjecture, one can find arguments that contain many of the same ideas in \cite{ACh} and \cite{Pspins} in the case of models with only even $p$-spin interactions, and a sketch of the proof of the general case in Section 15.3 in \cite{SG2}. The ingredient that was missing is the main result in \cite{PU} which also allows us to handle the case of the general mixed $p$-spin models.

\smallskip
\noindent
\textbf{The Ghirlanda-Guerra identities.} A central role in the proof is played by the Ghirlanda-Guerra identities \cite{GG} that are utilized in two distinct ways. First, they yield positivity of the overlap via Talagrand's positivity principle  which allows us to obtain the upper bound using Guerra's replica symmetry breaking interpolation scheme and, second, they imply ultrametricity of the overlap array using the main result in \cite{PU} which allows us to identify the asymptotic Gibbs measures that appear in the proof of the lower bound based on the Aizenman-Sims-Starr scheme \cite{AS2}. Let us consider a perturbation Hamiltonian 
\begin{equation}
H_N^{pert}(\vsi) = N^{-1/8} \sum_{p\geq 1} 2^{-p} x_p H_{N,p}'(\vsi),
\label{Hpert}
\end{equation}
where $H_{N,p}'(\vsi)$ are independent copies of the $p$-spin Hamiltonians  in (\ref{HamSKp}) and $(x_p)_{p\geq 1}$ are i.i.d. random variables uniform on an interval of length one, for example, $[1,2]$. Replacing $H_N$ with $H_N + H_N^{pert}$ in (\ref{Parisi}) obviously does not affect the limit since the perturbation term is of a smaller order. However, adding this perturbation term regularizes the Gibbs measure in the following way. Let $G_N$ be the Gibbs measure on $\Sigma_N$ corresponding to the Hamiltonian $H_N + H_N^{pert}$,
\begin{equation}
G_N(\vsi) = \frac{\exp(H_N(\vsi) + H_N^{pert}(\vsi))}{Z_N},\,\,
\end{equation}
where $Z_N = \sum_{\vsi\in\Sigma_N} \exp (H_N(\vsi)+H_N^{pert}(\vsi))$ and denote by $\la\cdot\ra$ the average with respect to the product Gibbs measure $G_N^{\otimes \infty}$. Let $(\vsi^l)_{l\geq 1}$ be an i.i.d. sequence of replicas sampled from $G_N$ and denote by 
\begin{equation}
R_{l,l'}=\frac{1}{N}\sum_{i\leq N} \sigma^l_i \sigma_i^{l'}
\label{overlap}
\end{equation}
the normalized scalar product, or overlap, of $\vsi^l$ and $\vsi^{l'}.$ Given $p\geq 1$, $n\geq 2$ and a bounded measurable function $f$ of the overlaps $(R_{l,l'})_{l,l'\leq n}$ on $n$ replicas, let 
\begin{equation}
\phi(f,n,p) = 
\Bigl|
\e_g  \la f R_{1,n+1}^p \ra -  \frac{1}{n}\e_g \la f \ra \hspace{0.3mm} \e_g\la R_{1,2}^p\ra - \frac{1}{n}\sum_{l=2}^{n}\e_g \la f R_{1,l}^p\ra
\Bigr|,
\label{GG}
\end{equation}
where $\e_g$ denotes  the expectation with respect to all Gaussian random variables for a fixed uniform sequence $(x_p)_{p\geq 1}$.
Then, the Ghirlanda-Guerra identities can be stated as follows.
\begin{proposition}\label{PropGG}
For any $p\geq 1, n\geq 2$ and a bounded function $f$ of the overlaps $(R_{l,l'})_{l,l'\leq n}$,
\begin{equation}
\lim_{N\to\infty} \e_x \hspace{0.3mm}\phi(f,n,p) = 0,
\label{GGxlim}
\end{equation} 
where $\e_x$ is the expectation with respect to $(x_p)_{p\geq 1}$. 
\end{proposition}\noindent
The proof of this result is well known and we refer to Chapter 12 in \cite{SG2} for details. We will not be using these identities directly for measure $G_N$ but for other Gibbs measures with a slightly modified Hamiltonian $H_{N}(\vsi)$ since it is well known that the proof of the identities is robust to such modifications and depends mostly on the  form of the perturbation Hamiltonian (\ref{Hpert}). It is interesting to note that once we finish the proof of Theorem \ref{Th1}, the argument in \cite{PGGmixed} will immediately imply that (\ref{GGxlim}) holds in a strong sense without the perturbation Hamiltonian for all $p\geq 1$ such that $\beta_p\not =0$ in (\ref{HN}).


\smallskip
\noindent \textbf{Guerra's replica symmetry breaking bound.} 
In the case when $p$-spin interactions for odd $p\geq 3$ are not present in (\ref{HN}), the inequality $\leq$ in (\ref{Parisi}) was proved by F. Guerra in \cite{Guerra} by inventing the replica symmetry breaking interpolation scheme. The fact that this inequality holds even in the presence of odd $p$-spin interactions was observed by M. Talagrand in \cite{TG} and we will only briefly remind the main idea, which is to write down Guerra's interpolation scheme in terms of the Ruelle probability cascades \cite{Ruelle} (Poisson-Dirichlet cascades in the terminology of \cite{SG2}) and force the overlap to be positive along  the interpolation by adding the perturbation term (\ref{Hpert}). Given $k\geq 1$, the Ruelle probability cascades are defined as (i) a random probability measure $(w_\alpha)_{\alpha\in \Natural^k}$ on $\Natural^k$ via some explicit construction involving Poisson-Dirichlet point processes $PD(\zeta)$ with intensity measures $x^{-1-\zeta}dx$ for $\zeta\in (0,1)$ and (ii) a Gaussian process $(z_\alpha)_{\alpha\in \Natural^k}$ with covariance $\e z_{\alpha^1} z_{\alpha^2} = \xi'(q_{\alpha^1\wedge \alpha^2})$ where 
$$
\mbox{
$\alpha^1\wedge\alpha^2 = \min\{l\geq 1: \alpha_l^1\not = \alpha_l^2\}$ if  $ \alpha^1\not=\alpha^2$ and  $\alpha^1\wedge\alpha^2 = k+1$ if $\alpha^1=\alpha^2$ 
}
$$
(see Chapter 14 in \cite{SG2} for details). For $0\leq t\leq 1$ we define an interpolating Hamiltonian
\begin{equation}
H_{N,t}(\vsi,\alpha) = 
\sqrt{t} H_N(\vsi) + \sqrt{1-t}\, \sum_{i\leq N} z_{\alpha, i} \sigma_i 
\label{Hta}
\end{equation}
where $(z_{\alpha,i})_{\alpha\in \Natural^k}$ are independent copies of $(z_\alpha)_{\alpha\in \Natural^k}$  for $i\geq 1$ and let
\begin{equation}
\varphi(t)=\frac{1}{N}\, \e\log \sum_{\alpha,\vsi} w_{\alpha} \exp \bigl( H_{N, t}(\vsi,\alpha) + H_N^{pert}(\vsi)\bigr).
\label{Gint}
\end{equation}
If we define the Gibbs measure $\Gamma_t$ on $\Sigma_N\times \mathbb{N}^{k}$ by 
$$
\Gamma_t\bigl\{(\vsi,\alpha)\bigr\} \sim w_{\alpha}\exp \bigl(H_{N,t}(\vsi,\alpha)+ H_N^{pert}(\vsi)\bigr)
$$
then a straightforward calculation using Gaussian integration by parts gives
\begin{equation}
\varphi'(t) =
-\frac{1}{2}\,\theta(1)
+\frac{1}{2}\,\e\bigl\la
\theta(q_{\alpha^1\wedge\alpha^2})
\bigr\ra_{\Gamma_t}
-\frac{1}{2}\,\e\bigl\la
\xi(R_{1,2})-R_{1,2}\xi'(q_{\alpha^1\wedge\alpha^2}) +\theta(q_{\alpha^1\wedge\alpha^2})
\bigr\ra_{\Gamma_t}
\label{phider2} 
\end{equation}
where $\la\cdot\ra_{\Gamma_t}$ is the Gibbs average with respect to $\Gamma_t^{\otimes 2}.$ When $\xi(x)=\sum_{p\geq 1}\beta_p^2 x^p$ does not contain terms for odd $p\geq 3$, $\xi$ is convex on $[-1,1]$, which implies that the last term in  (\ref{phider2}) is negative and dropping this term and integrating the corresponding inequality for $0\leq t\leq 1$ we obtain an upper bound on the free energy in (\ref{Parisi}). The fact that the representation of this upper bound in terms of the Ruelle probability cascades coincides with the formula in (\ref{Pk}) is well known and is explained in great detail in Chapter 14 in \cite{SG2}. If the terms for odd $p\geq 3$ are present, the function $\xi$ is only convex on $[0,1]$ but the argument still works if we know that $R_{1,2}$ is non-negative with high probability under $\e \Gamma_t^{\otimes 2}$. This is where the perturbation term in (\ref{Gint}) comes into play to ensure that the Ghirlanda-Guerra identities hold along the interpolation and, as a consequence, to ensure the positivity of the overlap via Talagrand's positivity principle (see Section 12.3 in \cite{SG2}). In fact, an observation in \cite{Posit} shows that the perturbation term $H_{N}^{pert}$ forces the positivity of the overlap uniformly over all measures on $\Sigma_N$ in the following sense. If given a measure $\nu_N$ on $\Sigma_N$ we define a random probability measure $\hat{\nu}_N$ on $\Sigma_N$ by the change of density $d\hat{\nu}_N(\vsi) \sim \exp H_N^{pert}(\vsi)\, d\nu_N(\vsi)$ then Theorem 1 in \cite{Posit} implies that for any $\eps>0$,
\begin{equation}
\lim_{N\to\infty}\sup_{\nu_N} \e\hspace{0.3mm} \hat{\nu}_N^{\otimes 2} (R_{1,2} \leq -\eps ) = 0.
\end{equation}
Using this for the marginal $\nu_N$ on $\Sigma_N$ of the Gibbs measure $\gamma_t\bigl\{(\vsi,\alpha)\bigr\} \sim w_{\alpha}\exp H_{N, t}(\vsi,\alpha)$ on $\Sigma_N\times \Natural^k$ implies that the remainder term in (\ref{phider2}) is asymptotically non-negative and we can proceed as in the case of even $p$-spin interactions.

\smallskip
\noindent
\textbf{The Aizenman-Sims-Starr scheme.} The proof of the lower bound is done in several steps but it begins with the Aizenman-Sims-Starr scheme \cite{AS2}. Let us consider the Hamiltonian $H_N^{-}(\vsi) = \sum_{p\geq 1} \beta_p H_{N,p}^{-}(\vsi)$ where 
\begin{equation}
H_{N,p}^{-}(\vsi)
=
\frac{1}{(N+1)^{(p-1)/2}}
\sum_{1\leq i_1,\ldots,i_p\leq N}g_{i_1,\ldots,i_p} \sigma_{i_1}\ldots\sigma_{i_p}.
\label{HamSKp2}
\end{equation}
Let $G_N^{-}$ and $\la\cdot\ra_{\hspace{-0.5mm}\mathunderscore}$ denote the Gibbs measure and its average corresponding to the Hamiltonian $H_N^{-} + H_N^{pert}$ and let $z(\vsi)$ and $y(\vsi)$ be two Gaussian processes on $\Sigma_N$ with covariances 
\begin{equation}
\e z(\vsi^1) z(\vsi^2) = \xi'(R_{1,2}),\,\, \e y(\vsi^1) y(\vsi^2) = \theta(R_{1,2})
\label{covzy}
\end{equation}
independent of each other and all other random variables. Then the Aizenman-Sims-Starr scheme in \cite{AS2} yields the following (see e.g. Section 15.8 in \cite{SG2}). 
\begin{proposition}\label{PropAS2}
The limit in (\ref{Parisi}) is bounded from below by
\begin{equation}
\liminf_{N\to\infty} \bigl(
\e \log \bigl \la \ch z(\vsi) \bigr \ra_{\hspace{-0.5mm}\mathunderscore}
- \e\log \bigl \la \exp y(\vsi) \bigr \ra_{\hspace{-0.5mm}\mathunderscore} 
\bigr).
\label{AS2}
\end{equation}
\end{proposition}\noindent
The only difference here is that we included the perturbation term $H_N^{pert}(\vsi)$ but, since it is of a smaller order, one can easily check that it does not affect the computation leading to this representation. Below, we will express the limit (\ref{AS2}) in terms of some asymptotic Gibbs measure that satisfies the exact form of the Ghirlanda-Guerra identities but, in order to do so, we first need to show that Propositions \ref{PropGG} and \ref{PropAS2} also hold with non-random choices of the sequence $x=(x_p)_{p\geq 1}$ (depending on $N$) rather than on average over $x$. We mentioned above that the proof of the Ghirlanda-Guerra identities is robust to modifications of the Hamiltonian $H_N$ and, in particular, they hold for the Gibbs measure $G_N^{-}$ so that if in (\ref{GG}) we replace $\la\cdot \ra$ by $\la\cdot \ra_{\hspace{-0.5mm}\mathunderscore}$ then (\ref{GGxlim}) still holds.
Let us consider a collection 
$$
\FF = \Bigl\{
(f,n,p) : p\geq 1, n\geq 2, f \mbox{ is a monomial of } (R_{l,l'})_{l,l'\leq n}
\Bigr\}.
$$
Since this a countable family we can enumerate it, $((f_j,n_j,p_j))_{j\geq 1},$ and define a function
\begin{equation}
\phi_\FF = \phi_\FF(x) = \sum_{j\geq 1} 2^{-j} \phi(f_j,n_j,p_j)
\label{FF}
\end{equation}
which depends on the variables in $x=(x_p)_{p\geq 1}$. Since each monomial $|f|\leq 1$, we can see from the definition (\ref{GG}) that $|\phi(f,n,p)|\leq 2$ and, therefore, the Ghirlanda-Guerra identities (\ref{GGxlim}) imply that $\e_x \phi_\FF \to 0$. Let
\begin{equation}
\lambda = \lambda(x)
=
\e_g \log \bigl \la \ch z(\vsi) \bigr \ra_{\hspace{-0.5mm}\mathunderscore}
- \e_g \log \bigl \la \exp y(\vsi) \bigr \ra_{\hspace{-0.5mm}\mathunderscore} 
\label{lambda}
\end{equation}
where, again, $\e_g$ denotes the expectation with respect to all Gaussian random variables for a fixed $x.$  We will need the following simple lemma.
\begin{lemma} We can find  $x=(x_p)_{p\geq 1}$ such that
\begin{equation}
\phi_\FF(x) \leq  c\hspace{0.3mm} (\e_x \phi_\FF)^{1/2} \,\mbox{ and }\,
\lambda(x) \leq \e_x \lambda +  c\hspace{0.3mm} (\e_x \phi_\FF)^{1/2},
\label{both}
\end{equation}
where $c$ is a constant that depends only on the function $\xi.$
\end{lemma}\noindent
\textbf{Proof.} If we denote by $\e_z$ and $\e_y$ the expectations with respect to $(z(\vsi))$ and $(y(\vsi))$ then (\ref{covzy}) and Jensen's inequality imply
$$
0\leq \e_g \log \bigl \la \ch z(\vsi) \bigr \ra_{\hspace{-0.5mm}\mathunderscore} \leq 
 \e_g \log \bigl \la \e_z \ch z(\vsi) \bigr \ra_{\hspace{-0.5mm}\mathunderscore} = \xi'(1)/2
$$
and
$$
0\leq \e_g \log \bigl \la \exp y(\vsi) \bigr \ra_{\hspace{-0.5mm}\mathunderscore} \leq 
 \e_g \log \bigl \la \e_y \exp y(\vsi) \bigr \ra_{\hspace{-0.5mm}\mathunderscore} = \theta(1)/2
$$
and, therefore, the function $-c\leq \lambda(x) \leq c.$ Given $\eps>0$, let $\Omega = \{x : \lambda(x) \leq \e_x \lambda +\eps\}$. Then, $\e_x \lambda \geq (\e_x \lambda +\eps) \p_x(\Omega^c) - c\hspace{0.3mm} \p_x(\Omega)$ and $\p_x(\Omega)\geq \eps/(\e_x\lambda + \eps+c)>\eps/c.$  On the other hand, by Chebyshev's inequality, $\p_x(\phi_\FF\leq \eps)\geq 1-\e_x \phi_\FF/\eps$ and if $\eps/c > \e_x \phi_\FF/\eps $ then $\Omega\cap \{\phi_\FF\leq \eps\}\not= \emptyset$. Taking $\eps = c \hspace{0.3mm}(\e_x \phi_\FF)^{1/2}$ implies that we can find a point $x$ that satisfies both inequalities in (\ref{both}).
\qed

\noindent
For each $N$, let us choose $x^N = (x_p^{N})_{p\geq 1}$ that satisfies (\ref{both}) and, since $\e_x \phi_\FF \to 0$, we get 
\begin{equation}
\lim_{N\to\infty}\phi_\FF(x^N) = 0 \,\,\mbox{ and }\,\,
\liminf_{N\to\infty} \e_x \lambda \geq \liminf_{N\to\infty} \lambda(x^N).
\label{xNlim}
\end{equation}
Let us redefine the Hamiltonian $H_N^{pert}$ and the Gibbs measure $G_N^{-}$ by fixing parameters $x=x^N$ and, since the measure now depends only on the Gaussian randomness, we will write $\e$ instead of $\e_g.$ By (\ref{xNlim}), Proposition \ref{PropAS2} still holds for this redefined measure $G_N^{-}$ and, recalling (\ref{FF}),
\begin{equation}
\e \la f R_{1,n+1}^p \ra_{\hspace{-0.5mm}\mathunderscore}  - \frac{1}{n} \e \la f  \ra_{\hspace{-0.2mm}\mathunderscore}  \hspace{0.2mm} \e \la  
R_{1,2}^p\ra_{\hspace{-0.2mm}\mathunderscore}  - \frac{1}{n}\sum_{l=2}^{n}\e \la f R_{1,l}^p\ra_{\hspace{-0.2mm}\mathunderscore} 
\to 0
\label{GGk}
\end{equation}
for all $p\geq 1, n\geq 2$ and all monomials $f$ of $(R_{l,l'})_{l,l'\leq n}$.



\smallskip
\noindent
\textbf{Asymptotic Gibbs' measures.} Next, we will define an asymptotic analogue of the Gibbs measure and represent the limit (\ref{AS2}) in terms of this measure. Let $(\vsi^l)_{l\geq 1}$ be an i.i.d. sample from  $G_N^{-}$ and let $R^N = (R_{l,l'}^N)_{l,l'\geq 1}$ be the normalized Gram matrix, or matrix of overlaps, of this sample.  Consider a subsequence $(N_k)$ along which the limit in (\ref{AS2}) is achieved (now with non-random parameters $x^N$) and the distribution of $R^N$ under ${\e G_N^{-}}^{\otimes \infty}$ converges in the sense of convergence of finite dimensional distributions to the distribution of some array $R^\infty$. For simplicity of notation, let us assume that the sequence $(N_k)$ coincides with natural numbers.  Under ${\e G_N^{-}}^{\otimes \infty}$,  the array $R^N$ is weakly exchangeable which means that  
\begin{equation}
(R^N_{\pi(l),\pi(l')}) \stackrel{d}{=} (R^N_{l,l'})
\end{equation}
for any permutation $\pi$ of finitely many indices. Obviously, this property will be preserved in the limit so that $R^\infty$ is a weakly exchangeable symmetric non-negative definite array and, following \cite{DS}, we will call any such array a Gram-de Finetti array. The Dovbysh-Sudakov representation \cite{DS} then guarantees that all such arrays are generated by i.i.d. samples from random measures on a separable Hilbert space (see \cite{PDS} for a detailed proof).
\begin{proposition} \label{ThDS} 
If $(R_{l,l'})_{l,l'\geq 1}$ is a Gram-de Finetti array such that $R_{l,l'} = 1$ then there exists a random measure $G$ on the unit ball of a separable Hilbert space such that
\begin{equation}
\bigl(R_{l,l'}\bigr)_{l,l'\geq 1} 
\stackrel{d}{=}
\bigl(\vrho^l\cdot\vrho^{l'} + \delta_{l,l'}(1-\|\vrho^l\|^2) \bigr)_{l,l'\geq 1}
\label{DS}
\end{equation}
where $(\vrho^l)$ is an i.i.d. sample from $G$.
\end{proposition}\noindent
The importance of the Dovbysh-Sudakov representation in spin glass models was first clearly demonstrated in \cite{AA} and other examples where this representation played an important role can be found in \cite{ACh}, \cite{PGG} and \cite{PGGsimple}. Let $G$ be a random measure generating the array $R^\infty$, let $(\vrho^l)$ be an i.i.d. sample from $G$ and let $R_{l,l'} = \vrho^l\cdot\vrho^{l'}$ for $l\not = l'$ and $R_{l,l}=1$. For simplicity of notation, we will now omit $\infty$ in $R^\infty$.  If we denote by $\la\cdot\ra$ the average with respect to $G$ then, by (\ref{GGk}), the measure $G$ satisfies the Ghirlanda-Guerra identities, 
\begin{equation}
\e  \la f R_{1,n+1}^p \ra = \frac{1}{n}  \e \la f \ra \hspace{0.3mm} \e \la R_{1,2}^p\ra 
+\frac{1}{n} \sum_{l=2}^{n}\e  \la f R_{1,l}^p \ra,
\label{GGlimit}
\end{equation}
for all $p\geq 1, n\geq 2$ and all monomials $f$ of $(R_{l,l'})_{l,l'\leq n}$. Approximating bounded functions of the overlaps in $L^1$ by polynomials we also have
\begin{equation}
\e  \la f \psi(R_{1,n+1}) \ra = \frac{1}{n}  \e \la f \ra \hspace{0.3mm} \e \la \psi(R_{1,2})\ra 
+\frac{1}{n} \sum_{l=2}^{n}\e  \la f \psi(R_{1,l}) \ra 
\label{GGgen}
\end{equation}
for bounded measurable functions $f$ and $\psi$. Below, the identities (\ref{GGgen}) will allow us to identify these asymptotic Gibbs measures but, first, let us  show how the limit in (\ref{AS2}) can be represented in terms of $G$. By Theorem 2 in \cite{PGG}, (\ref{GGgen}) implies that if $q^*$ is the largest point in the support of the distribution of $R_{1,2}$ under $\e G^{\otimes 2}$ then $G$ is concentrated on the sphere of radius $\sqrt{q^*}$ with probability one and, therefore, $R$ is generated by $(\vrho^l\cdot\vrho^{l'} + \delta_{l,l'}(1-q^*))_{l,l'\geq 1}$. Let  $z(\vrho)$ and $y(\vrho)$ be two Gaussian processes on the unit ball of our Hilbert space with covariances 
\begin{equation}
\e z(\vrho^1) z(\vrho^2) = \xi'(\vrho^1\cdot \vrho^2),\,\, \e y(\vrho^1) y(\vrho^2) = \theta(\vrho^1\cdot\vrho^2),
\label{zy}
\end{equation}
let $\eta$ be a standard Gaussian random variable independent of everything else and let $\e_\eta$ denote the expectation in $\eta$ only. Then the following holds.
\begin{lemma}\label{ThG}
We have
\begin{equation}
\lim_{N\to\infty} \e\log \la \ch z(\vsi)\ra_{\hspace{-0.5mm}\mathunderscore} = 
\e \log \e_\eta \bigl \la \ch\hspace{-0.3mm} (z(\vrho) +\eta (\xi'(1)-\xi'(q^*))^{1/2} ) \bigr \ra
\label{limAS1}
\end{equation}
and
\begin{equation}
\lim_{N\to\infty} \e\log \la \exp y(\vsi)\ra_{\hspace{-0.5mm}\mathunderscore}  = 
\e \log \e_\eta \bigl \la \exp\hspace{0.1mm} (y(\vrho) +\eta (\theta(1)-\theta(q^*))^{1/2})\bigr \ra.
\label{limAS2}
\end{equation}
\end{lemma}\noindent
The proof of Lemma \ref{ThG} is based on the following observation.  For a moment, let $R=(R_{l,l'})_{l,l'\geq 1}$ be an arbitrary Gram-de Finetti array such that $R_{l,l'} = 1$, let $\LL$ be its distribution and let $G$ be any random measure generating $R$ as in Proposition \ref{ThDS}. It is known that in some sense this measure is unique but we will not need it here. Let us define
\begin{equation}
\Phi(\LL)
=
\e \log \e_\eta \bigl \la \ch\hspace{-0.3mm} \bigl(z(\vrho) +\eta (\xi'(1)-\xi'(\|\vrho \|^2))^{1/2} \bigr) \bigr \ra.
\label{Phi}
\end{equation}
The Gaussian process $z(\vrho)$ here is the same as in (\ref{zy}) but we do not assume now that $G$ is concentrated on the sphere $\|\vrho\|^2 = q^*$. We will prove that the right hand side in (\ref{Phi}) does not depend on the choice of the measure $G$ and, indeed, depends only on the distribution $\LL$ in a continuous fashion.
\begin{lemma}\label{LemPhi}
The function $\LL\to \Phi(\LL)$ defined in (\ref{Phi}) is continuous with respect to weak convergence of the distribution $\LL$ .
\end{lemma}\noindent
\textbf{Proof of Lemma \ref{ThG}.} Since $R^N$ is the Gram matrix of the sequence $(N^{-1/2}\vsi^l)$, we can simply think of the measure $G_N^{-}$ as defined on $N^{-1/2}\Sigma_N$ which is a subset of the sphere $\| \vsi\|=1$ in $\Reals^N$. Then (\ref{covzy}) agrees with (\ref{zy}) and Lemma \ref{LemPhi} implies (\ref{limAS1}) since $R^N$ converges in distribution to $R^\infty$ and, as we mentioned above, the Ghirlanda-Guerra identities (\ref{GGlimit}) imply that $G$ is concentrated on the sphere $\|\vrho\|^2=q^*$. Equation (\ref{limAS2}) can be proved similarly.
\qed

\noindent\textbf{Proof of Lemma \ref{LemPhi}.}
The proof is almost identical to the proof of Lemma 11 in \cite{Pspins}. For simplicity of notation, let us denote 
$$
z_\eta(\vrho) = z(\vrho) +\eta (\xi'(1)-\xi'(\|\vrho\|^2))^{1/2}
$$ 
and let $\e_z$ be the expectation in the randomness of $(z(\vrho))$ conditionally on all other random variables. By standard concentration inequalities for Gaussian processes (see, for example, Lemma 3 in \cite{Posit}) we have that for $a\geq 1$,
\begin{equation}
\p_z\bigl(\bigl|  \log \e_\eta \la \ch z_\eta (\vrho)\ra - \e_z \log \e_\eta \la \ch z_\eta(\vrho)\ra \bigr| 
\geq a\bigr)\leq \exp (-ca^2)
\label{GaussC}
\end{equation}
for some small enough constant $c$ that depends only on the function $\xi$ through (\ref{zy}). Since
$$
0
\leq
\e_z  \log \e_\eta \la \ch z_\eta(\vrho)\ra 
\leq
\log \la \e_z \e_\eta \ch z_\eta(\vrho)\ra = \xi'(1)/2,
$$ 
the inequality (\ref{GaussC}) implies that $\p (  |\log \e_\eta \la \ch z_\eta(\vrho)\ra |  \geq a)\leq \exp(-ca^2)$ for small $c$ and large enough $a$ and, therefore, if we denote $\log_a x = \max(-a,\min(\log x, a))$ then for large $a,$
\begin{equation}
\bigl|
\e \log \e_\eta \la \ch z_\eta (\vrho)\ra
-
\e\log_a  \e_\eta \la  \ch z_\eta (\vrho)\ra
\bigr| 
\leq \exp(-ca^2).
\label{partA}
\end{equation}
Next, if we define  $\ch_a\hspace{0.3mm} x = \min(\ch x,\ch a)$ then using  that $|\log_a x - \log_a y | \leq e^a |x-y|$ and $| \ch x - \ch_a\hspace{0.3mm}  x | \leq \ch x\, I(|x| \geq a)$ we can write
\begin{eqnarray*}
\bigl|
\e\log_a  \e_\eta \la \ch z_\eta (\vrho)\ra
-
\e\log_a \e_\eta\la \ch_a\hspace{0.3mm}  z_\eta(\vrho)\ra
\bigr| 
&\leq &
e^a  \hspace{0.3mm}
\e \bigl\la
\bigl|
\ch z_\eta(\vrho)
-
\ch_a\hspace{0.3mm}  z_\eta(\vrho)
\bigr|
\bigr \ra
\\
&\leq&
e^a  \hspace{0.3mm}
\e \bigl\la
\ch z_\eta(\vrho)\,
I(|z_\eta(\vrho)| \geq a)
\bigr \ra.
\end{eqnarray*}
By H\"older's inequality, this can be bounded by
$$
e^a \hspace{0.3mm}
\bigl(
\e \bigl\la
\e_{z,\eta} \ch^2 z_\eta(\vrho)
\bigr\ra
\bigr)^{1/2}\,
\bigl(
\e \bigl\la
\p_{z,\eta} (|z_\eta(\vrho)| \geq a)
\bigr \ra
\bigr)^{1/2}
\leq \exp(-ca^2)
$$
for small $c$ and large enough $a$ since  $\p_{z,\eta} (|z_\eta(\vrho)|  \geq a)\leq \exp(-ca^2).$
Combining with (\ref{partA}),
\begin{equation}
\bigl|
\e \log \e_\eta \la \ch z_\eta (\vrho)\ra
-
\e\log_a  \e_\eta \la  \ch_a\hspace{0.3mm} z_\eta (\vrho)\ra
\bigr| 
\leq \exp(-ca^2).
\label{partA2}
\end{equation} 
Approximating logarithm by polynomials on the interval $[e^{-a},e^a]$, $\e\log_a \e_\eta \la \ch_a\hspace{0.3mm} z_\eta (\vrho)\ra$ can be approximated by a linear combinations of moments
\begin{equation}
\e \bigl(\e_\eta \la \ch_a\hspace{0.3mm} z_\eta(\vrho)\ra \bigr)^r
=
\e \bigl\la \e_z \e_\eta \prod_{l\leq r} \ch_a (z_{\eta^l}(\vrho^l)) \bigr\ra
\label{preF}
\end{equation}
where we used replicas and where $z_{\eta^l}(\vrho^l) = z(\vrho^l) + \eta^l (\xi'(1)-\xi'(\|\vrho^l\|^2))^{1/2}$ and $(\eta^l)$ are i.i.d. standard Gaussian. Since the covariance of the Gaussian sequence $(z_{\eta^l}(\vrho^l))$ is equal to
$$
\xi'(\vrho^l\cdot\vrho^{l'}) + \delta_{l,l'} (\xi'(1)-\xi'(\|\vrho^l\|^2)) = \xi'(R_{l,l'}),
$$ 
the function inside the Gibbs average on the right hand side of (\ref{preF}) is equal to
\begin{equation}
\e_z \e_\eta \prod_{l\leq r} \ch_a (z_{\eta^l}(\vrho^l))
=
F((R_{l,l'})_{l,l'\leq r})
\label{Fdef}
\end{equation}
for some continuous bounded function $F$ of the overlaps $(R_{l,l'})_{l,l'\leq r}$. Together with (\ref{partA2}) this shows that we can approximate $\Phi(\LL)$ arbitrarily well by a linear combination of $\e\la F(R)\ra$ for some continuous bounded functions $F$ of finitely many overlaps, which proves that $\Phi(\LL)$ is continuous with respect to the distribution $\LL$ of the overlap array $R$.
\qed


\smallskip
\noindent
\textbf{Identifying asymptotic Gibbs' measures using ultrametricity.} To show that the lower bound in (\ref{AS2}) matches Guerra's upper bound, it remains to identify the difference of (\ref{limAS1}) and (\ref{limAS2}) with the functional (\ref{Pk}). Since the asymptotic Gibbs measure $G$ satisfies the Ghirlanda-Guerra identities (\ref{GGgen}), the main result in \cite{PU} implies that the support of $G$ is ultrametric with probability one, that is,
\begin{equation}
\e \bigl\la I\bigl(R_{1,2}\geq \min(R_{1,3},R_{2,3})\bigr) \bigr\ra=1.
\label{ultra}
\end{equation}
Given $k\geq 1$, let us consider a function $\kappa(q)$ on $[0,1]$ such that 
\begin{equation}
\mbox{$\kappa(q) = j/k\,$ for $\, j/k\leq q<(j+1)/k$, $j=0,\ldots, k.$}
\label{finite}
\end{equation}
Equation (\ref{ultra}) implies that for any $q$ the inequality $q\leq \vrho^l\cdot\vrho^{l'}$ defines an equivalence relation $l\sim l'$ and, therefore, the array $(I(q\leq R_{l,l'}))_{l,l'\geq 1}$ is non-negative definite since it is block-diagonal with blocks consisting of all elements equal to one. This implies that $R^\kappa = (\kappa(R_{l,l'}))_{l,l'\geq 1}$ is non-negative definite since it can be written as a convex combination
$$
\kappa(R_{l,l'}) = \sum_{j=1}^{k} \frac{1}{k}I\Bigl(\frac{j}{k}\leq R_{l,l'}\Bigr).
$$ 
In addition, it is clear that $R^\kappa$ is  weakly exchangeable and satisfies the Ghirlanda-Guerra identities (\ref{GGgen}). Then, by the Dovbysh-Sudakov representation (\ref{DS}), $R^\kappa$ can be generated by a sample from some random measure $G^\kappa$ on the unit ball of a Hilbert space. If for simplicity we assume that $q^*\not = j/k$ for $j\leq k$ then $\kappa(q^*)$ is the largest point in the support of the distribution of $\kappa(R_{1,2})$ and by Theorem 2 in \cite{PGG} the measure $G^\kappa$ is concentrated on the sphere $\|\vrho\|^2 = \kappa(q^*)$. When $k\to\infty$, the distribution of $R^\kappa$ converges weakly to the distribution of $R$ and if we denote by $\la\cdot\ra_\kappa$ the average with respect to the measure $G^\kappa$ then Lemma \ref{LemPhi} implies that 
\begin{equation}
\e \log \e_\eta \bigl \la \ch\hspace{-0.3mm} (z(\vrho) +\eta (\xi'(1)-\xi'(\kappa(q^*))^{1/2} ) \bigr \ra_\kappa
\label{fterm}
\end{equation}
approximates the right hand side of (\ref{limAS1}). It is well known that an ultrametric measure, such as $G^\kappa$, 
that satisfies the Ghirlanda-Guerra identities and under which the overlaps $R^\kappa$ take finitely many values as in (\ref{finite}) can be identified with the discrete Ruelle probability cascades by the Baffioni-Rosati theorem \cite{BR} (see the proof of Theorem 15.3.6 in \cite{SG2} for details). The fact that in this case (\ref{fterm}) coincides with $X_0(\vec{m},\vec{q})$ in (\ref{Pk}) with parameters $q_j = j/k$ and $m_j = \e\la I(R_{1,2}\leq q_j)\ra$ is also well known (see e.g. Theorem 14.2.1 in \cite{SG2}). One can similarly show that (\ref{limAS2}) corresponds to the second term in (\ref{Pk}) and this finishes the proof of Theorem \ref{Th1}. One could also work with the continuous Ruelle probability cascades using a general theory developed in \cite{Bolthausen} but, at this point, it was easier to simply discretize the overlap array and Gibbs measure.
\qed



\begin{thebibliography}{99}
\footnotesize


\bibitem{AS2} Aizenman, M., Sims, R., Starr, S. (2003)
An extended variational principle for the SK spin-glass model.
{\it Phys. Rev. B,} \textbf{68}, 214403.  

\bibitem{AA} Arguin, L.-P., Aizenman, M. (2009)
On the structure of quasi-stationary competing particles systems. 
{\it Ann. Probab.}, \textbf{37}, no. 3, 1080-1113. 

\bibitem{ACh} Arguin, L.-P., Chatterjee, S. (2010)
Random overlap structures: properties and applications to spin glasses.
Preprint, arxiv:1011.1823.

\bibitem{BR} Baffioni, F., Rosati, F. (2000) 
Some exact results on the ultrametric overlap distribution in mean field spin glass models.
{\it Eur. Phys. J. B}, \textbf{17}, 439-447. 

\bibitem{Bolthausen} Bolthausen, E., Sznitman, A.-S. (1998) On Ruelle's probability cascades and an abstract cavity method. {\it Comm. Math. Phys.}, \textbf{197}, no. 2, 247-276.

\bibitem{DS} Dovbysh, L. N., Sudakov, V. N. (1982)  Gram-de Finetti matrices. 
{\it Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov.},  \textbf{119}, 77-86. 

\bibitem{GG} Ghirlanda, S., Guerra, F. (1998) 
General properties of overlap probability distributions in disordered spin systems. 
Towards Parisi ultrametricity.  {\it J. Phys. A},  \textbf{31}, no. 46, 9149-9155.

\bibitem{Guerra} Guerra, F. (2003) Broken replica symmetry bounds in the mean field spin glass model.
{\it Comm. Math. Phys.}, \textbf{233}, no. 1, 1-12.

\bibitem{M1} M\'ezard, M., Parisi, G., Sourlas, N., Toulouse, G., Virasoro, M.A. (1984) On the nature of the spin-glass phase.
{\it Phys. Rev. Lett.}, \textbf{52}, 1156.  

\bibitem{M2} M\'ezard, M., Parisi, G., Sourlas, N., Toulouse, G., Virasoro, M.A.  (1984) Replica symmetry breaking and the nature of the spin-glass phase. {\it J. de Physique}, \textbf{45}, 843.

\bibitem{Posit} Panchenko, D. (2007) A note on Talagrand's positivity principle.  
{\it Electron. Comm. Probab.},  \textbf{12}, 401-410. 

\bibitem{PGG} Panchenko, D. (2010) A connection between Ghirlanda-Guerra identities and ultrametricity. 
{\it Ann. of Probab.}, \textbf{38}, no. 1, 327-347. 

\bibitem{PDS} Panchenko, D. (2010) On the Dovbysh-Sudakov representation result.
{\it Electron. Comm. in Probab.},  \textbf{15}, 330-338. 

\bibitem{PGGmixed} Panchenko, D. (2010) The Ghirlanda-Guerra identities for mixed $p$-spin model.
{\it C. R. Acad. Sci. Paris,} Ser. I, \textbf{348}, 189-192. 

\bibitem{Pspins} Panchenko, D. (2010) Spin glass models from the point of view of spin distributions.
To appear in the {\it Ann. of Probab.}

\bibitem{PGGsimple} Panchenko, D. (2011) Ghirlanda-Guerra identities and ultrametricity: An elementary proof in the discrete case. {\it C. R. Acad. Sci. Paris,} Ser. I, \textbf{349}, 813-816.

\bibitem{PU} Panchenko, D. (2011) The Parisi ultrametricity conjecture. arXiv:1112.1003.

\bibitem{Parisi79} Parisi, G. (1979) Infinite number of order parameters for spin-glasses.
{\it Phys. Rev. Lett.}, \textbf{43}, 1754-1756.

\bibitem{Parisi} Parisi, G. (1980) A sequence of approximate solutions to the 
S-K model for spin glasses. {\it J. Phys. A}, \textbf{13}, L-115.

\bibitem{Ruelle} Ruelle, D. (1987) A mathematical reformulation of Derrida's REM and GREM.  
{\it Comm. Math. Phys.},  {\bf 108},  no. 2, 225-239.

\bibitem{SK} Sherrington, D., Kirkpatrick, S. (1972) Solvable model of a spin glass. 
{\it Phys. Rev. Lett.}, {\bf 35}, 1792-1796.

\bibitem{SG} Talagrand, M. (2003)   Spin Glasses: a Challenge for Mathematicians.  
Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge A Series of Modern Surveys in Mathematics, Vol. 43.
Springer-Verlag.

\bibitem{TG} Talagrand, M. (2003)  On Guerra's broken replica-symmetry bound. 
{\it C. R. Math. Acad. Sci. Paris}, {\bf 337}, no. 7, 477-480.

\bibitem{T-P} Talagrand, M. (2006) The Parisi formula. {\it Ann. of Math. (2)},  \textbf{163},  no. 1, 221-263.

\bibitem{SG2} Talagrand, M. (2011) Mean-Field Models for Spin Glasses. 
Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge A Series of Modern Surveys in Mathematics, Vol. 54, 55.
Springer-Verlag.





\end{thebibliography}





\end{document}

invlat.tex


%These are definitions for the preamble of TEX files

%\loadmsbm
%that may be needed for \Bbb (``blackboard bold, Bourbaki'') characters
%\UseAMSsymbols
%
%
%\magnification = \magstep1
\font\tencmmib=cmmib10 \skewchar\tencmmib '60
\newfam\cmmibfam
\textfont\cmmibfam=\tencmmib
\def\Ph{\mathchar'4010}
\def\Ps{\mathchar'4011}
\font\tensmc=cmcsc10
\def\smc{\tensmc}
\def\srm{\sevenrm }
%\font\teneufm=EUFM10
%\font\eighteufm=EUFM10 scaled 800
%\def\Fff#1{\hbox{\teneufm#1}}
%\def\Ffff#1{\hbox{\eighteufm#1}}
\font\tenmsb=msbm10
\font\eightmsb=msbm10 scaled 800
\def\Bbb#1{\hbox{\tenmsb#1}}
\def\Bbbb#1{\hbox{\eightmsb#1}}
%\font\bb=msbm10
%\def{\hbox{\bb N}}
%\def\RR{\hbox{\bb R}}
%\def \NN {{I\kern-.3em N}}
%\def \RR {{I\kern-.3em R}}
\def\bbox{\quad\hbox{\vrule \vbox{\hrule \vskip2pt \hbox{\hskip2pt
\vbox{\hsize=1pt}\hskip2pt} \vskip2pt\hrule}\vrule}}
\def\lessim{\ \lower4pt\hbox{$
\buildrel{\displaystyle <}\over\sim$}\ }
\def\gessim{\ \lower4pt\hbox{$\buildrel{\displaystyle >}
\over\sim$}\ }
\def\n{\noindent}
%\def\P{{\rm P}}
%\def\E{{\rm E}}
%\def\LL{{\cal L}}
\def\n{\noindent}
%\parskip=0.03truein
%\hfuzz=0.2truein
%
%\scriptfont4 = \sevensy
%\scriptscriptfont4 = \fivesy
%\font \gross=cmbx10 scaled \magstep2
%\font \mittel=cmbx10 scaled \magstep0  
%\font \outline=ocrb10f scaled \magstep1 
%\pageno = 1
\def\AA{{\cal A} }
\def\AAA{{\bf A} }
\def\Aap{(A,\AA ,P)}
\def\alf{\alpha}
\def\anorm{\|^.\|}
\def\AP{\it Ann. Probab. \bf}
\def\bbe{\vskip .15pc\hangindent=.5pc\hangafter=1}
\def\BB{{\cal B} }
\def\bxt{\boxtimes}
\def\CC{{\cal C} }
\def\claw{\Rightarrow}
\def\cntr{\centerline}
\def\cosph{\cos\phi}
\def\costh{\cos\theta}
\def\DAN{Dokladi AN Ukr.SSR}
\def\DD{{\cal D} }
\def\dc{{\buildrel {d}\over \longrightarrow}}
\def\defn{\skb\fl Definition\rm. }
\def\dens{\text{dens}}
\def\densC{\dens(\CC)}
\def\Dl{\Delta}
\def\dl{\delta}
\def\dnar{\downarrow}
\def\dpo{d_{P,1}}
\def\dpno{d_{P_n,1}}
\def\dpt{d_{P,2}}
\def\EE{{\Bbb E} }
\def\eee{\ =\ }
\def\eeq{\ \equiv \ }
\def\endp{\vfill\break}
\def\Eng{English transl.:}
\def\eps{\varepsilon}
\def\FF{{\cal F} }
\def\GG{{\cal G} }
\def\gee{\ \ge \ }
\def\goin{\to \infty}
\def\go0{\to 0}
\def\grad{\triangledown}
\def\fl{\smallskip}
%\def\fl{\flushpar} 
\def\HH{{\cal H} }
\def\inv{^{-1}}
\def\JJ{{\cal J} }
\def\KK{{\cal K} }
\def\la{\langle}
\def\li{< \infty}
\def\linF{\ell\upin(\FF)\ }
\def\lm{\lambda}
\def\Lm{\Lambda}
\def\lee{\ \le \ }
\def\leftitem#1{\item{\hbox to\parindent{\enspace#1\hfill}}}
\def\LL{{\cal L} }
\def\Lt{L^2(P)}
\def\Lto{L^2_0(P)}
\def\mc{m^{\CC }(}
\def\mcn{m^{\CC}(n)}
\def\mdsk{\medskip}
\def\mm{\vskip 1mm}
\def\MM{{\cal M} }
\def\nF{\|_{\FF}}
\def\NN{{\cal N} }
\def\normf{\|\cdot\|_{\cal F}}
\def\omg{\omega}
\def\Omg{\Omega}
\def\Omgap{(\Omg,\AA,{\rm Pr})\ }
\def\Omgsp{(\Omg, \Sigma ,{\rm Pr})\ }
\def\Omgspb{(\hat \Omg, \hat \Sigma ,\hat Pr)\ }
\def\Omgspr{(\Omg_{\eps }, \Sigma_{\eps } ,Pr_{\eps })\ }
\def\Omgspp{(\Omg_{\nu }, \Sigma_{\nu } , Pr_{\nu })\ }
\def\OO{{\bf O}}
\def\pard{\partial}
\def\pf{\hfill\break\bf Proof. \rm}
\def\pl{\ + \ }
\def\PP{{\cal P} }
\def\pr{probability}
\def\prm{^{\prime}}
\def\prc{{\buildrel {p}\over \longrightarrow}}
\def\prloc{{\buildrel {p,loc}\over \longrightarrow}}  
\def\prbloc{{\buildrel {{p, bloc}}\over \longrightarrow}}  
\def\qed{\hfill\break\rightline{$\bbox$}}
\def\QQ{{\cal Q}}
\def\ra{\rangle} 
\def\rbar{\overline{\Bbb R}}
\def\refs{\mdsk\cntr {REFERENCES}\mdsk}
\def\RR{{\Bbb R }}
\def\rtar{\Rightarrow}
\def\Ru{(in Russian)\ }
\def\sap{(S,\AA ,P)}
\def\soc{S(\CC )}
\def\setm{\setminus}
\def\sg{\sigma}
\def\sga{$\sigma$-algebra }
\def\sgal{$\sigma$-algebra}
\def\sg2{\sigma^2}
\def\sinph{\sin\phi}
\def\sinth{\sin\theta}
\def\skb{\medskip\fl\bf}
\def\sqt{^{1/2}}
\def\Sp{{\cal SP}(\RR^d)}
\def\SS{{\cal S} }
\def\SSSi{\smallint^{\infty}}
\def\SSOI{\smallint^1_0}
\def\SSS{\smallint}
\def\SSSW{\smallint^{\infty}_{-\infty}}
\def\tens{\otimes}
\def\TVMS{Teoria verojatnostej i matemati\v ceskaia statistika }
\def\TPA{\it Theor. Probability Appl. \bf}
\def\TPMS{Probability Theory and Mathematical Statistics, American
Mathematical Society} 
\def\tpr{\text{Pr}}
\def\TT{{\cal T} }
\def\subin{\_{\infty}}
\def\sumi{\sum^{\infty}}
\def\UMZ{Ukrainskij matemati\v ceskij \v zurnal}
\def\upar{\uparrow}
\def\upin{^{\infty}}
\def\VV{{\cal V}} 
\def\vc{Vapnik-\v Cervonenkis\ }
\def\wc{{\buildrel {w}\over \longrightarrow}}
\def\wloc{{\buildrel {w,loc}\over \longrightarrow}}  
\def\wbloc{{\buildrel {w, bloc }\over \longrightarrow}} 
\def\WW{{\cal W} } 
\def\XX{{\bf X} }
\def\xa{(\XX,\AA )\ }
\def\xap{(\XX,\AA,P)\ }
\def\xy{\{x,y\}}
\def\ZW{\it Z. Wahrscheinlichkeitsth. verw. Geb. \bf}
\def\ZZ{{\cal Z} }
\def\__{_{\infty}}
%\medskipamount=6pt plus 2pt minus 2pt










1112.4409.extracted_bib
\begin{thebibliography}{99}
\footnotesize


\bibitem{AS2} Aizenman, M., Sims, R., Starr, S. (2003)
An extended variational principle for the SK spin-glass model.
{\it Phys. Rev. B,} \textbf{68}, 214403.  

\bibitem{AA} Arguin, L.-P., Aizenman, M. (2009)
On the structure of quasi-stationary competing particles systems. 
{\it Ann. Probab.}, \textbf{37}, no. 3, 1080-1113. 

\bibitem{ACh} Arguin, L.-P., Chatterjee, S. (2010)
Random overlap structures: properties and applications to spin glasses.
Preprint, arxiv:1011.1823.

\bibitem{BR} Baffioni, F., Rosati, F. (2000) 
Some exact results on the ultrametric overlap distribution in mean field spin glass models.
{\it Eur. Phys. J. B}, \textbf{17}, 439-447. 

\bibitem{Bolthausen} Bolthausen, E., Sznitman, A.-S. (1998) On Ruelle's probability cascades and an abstract cavity method. {\it Comm. Math. Phys.}, \textbf{197}, no. 2, 247-276.

\bibitem{DS} Dovbysh, L. N., Sudakov, V. N. (1982)  Gram-de Finetti matrices. 
{\it Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov.},  \textbf{119}, 77-86. 

\bibitem{GG} Ghirlanda, S., Guerra, F. (1998) 
General properties of overlap probability distributions in disordered spin systems. 
Towards Parisi ultrametricity.  {\it J. Phys. A},  \textbf{31}, no. 46, 9149-9155.

\bibitem{Guerra} Guerra, F. (2003) Broken replica symmetry bounds in the mean field spin glass model.
{\it Comm. Math. Phys.}, \textbf{233}, no. 1, 1-12.

\bibitem{M1} M\'ezard, M., Parisi, G., Sourlas, N., Toulouse, G., Virasoro, M.A. (1984) On the nature of the spin-glass phase.
{\it Phys. Rev. Lett.}, \textbf{52}, 1156.  

\bibitem{M2} M\'ezard, M., Parisi, G., Sourlas, N., Toulouse, G., Virasoro, M.A.  (1984) Replica symmetry breaking and the nature of the spin-glass phase. {\it J. de Physique}, \textbf{45}, 843.

\bibitem{Posit} Panchenko, D. (2007) A note on Talagrand's positivity principle.  
{\it Electron. Comm. Probab.},  \textbf{12}, 401-410. 

\bibitem{PGG} Panchenko, D. (2010) A connection between Ghirlanda-Guerra identities and ultrametricity. 
{\it Ann. of Probab.}, \textbf{38}, no. 1, 327-347. 

\bibitem{PDS} Panchenko, D. (2010) On the Dovbysh-Sudakov representation result.
{\it Electron. Comm. in Probab.},  \textbf{15}, 330-338. 

\bibitem{PGGmixed} Panchenko, D. (2010) The Ghirlanda-Guerra identities for mixed $p$-spin model.
{\it C. R. Acad. Sci. Paris,} Ser. I, \textbf{348}, 189-192. 

\bibitem{Pspins} Panchenko, D. (2010) Spin glass models from the point of view of spin distributions.
To appear in the {\it Ann. of Probab.}

\bibitem{PGGsimple} Panchenko, D. (2011) Ghirlanda-Guerra identities and ultrametricity: An elementary proof in the discrete case. {\it C. R. Acad. Sci. Paris,} Ser. I, \textbf{349}, 813-816.

\bibitem{PU} Panchenko, D. (2011) The Parisi ultrametricity conjecture. arXiv:1112.1003.

\bibitem{Parisi79} Parisi, G. (1979) Infinite number of order parameters for spin-glasses.
{\it Phys. Rev. Lett.}, \textbf{43}, 1754-1756.

\bibitem{Parisi} Parisi, G. (1980) A sequence of approximate solutions to the 
S-K model for spin glasses. {\it J. Phys. A}, \textbf{13}, L-115.

\bibitem{Ruelle} Ruelle, D. (1987) A mathematical reformulation of Derrida's REM and GREM.  
{\it Comm. Math. Phys.},  {\bf 108},  no. 2, 225-239.

\bibitem{SK} Sherrington, D., Kirkpatrick, S. (1972) Solvable model of a spin glass. 
{\it Phys. Rev. Lett.}, {\bf 35}, 1792-1796.

\bibitem{SG} Talagrand, M. (2003)   Spin Glasses: a Challenge for Mathematicians.  
Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge A Series of Modern Surveys in Mathematics, Vol. 43.
Springer-Verlag.

\bibitem{TG} Talagrand, M. (2003)  On Guerra's broken replica-symmetry bound. 
{\it C. R. Math. Acad. Sci. Paris}, {\bf 337}, no. 7, 477-480.

\bibitem{T-P} Talagrand, M. (2006) The Parisi formula. {\it Ann. of Math. (2)},  \textbf{163},  no. 1, 221-263.

\bibitem{SG2} Talagrand, M. (2011) Mean-Field Models for Spin Glasses. 
Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge A Series of Modern Surveys in Mathematics, Vol. 54, 55.
Springer-Verlag.





\end{thebibliography}