  The aim of generalized phase retrieval is to recover $\mathbf{x}\in
\mathbb{F}^d$ from the quadratic measurements
$\mathbf{x}^*A_1\mathbf{x},\ldots,\mathbf{x}^*A_N\mathbf{x}$, where $A_j\in
\mathbf{H}_d(\mathbb{F})$ and $\mathbb{F}=\mathbb{R}$ or $\mathbb{C}$. In this
paper, we study the matrix set $\mathcal{A}=(A_j)_{j=1}^N$ which has the almost
everywhere phase retrieval property. For the case $\mathbb{F}=\mathbb{R}$, we
show that $N\geq d+1$ generic matrices with prescribed ranks have almost
everywhere phase retrieval property. We also extend this result to the case
where $A_1,\ldots,A_N$ are orthogonal matrices and hence establish the almost
everywhere phase retrieval property for the fusion frame phase retrieval. For
the case where $\mathbb{F}=\mathbb{C}$, we obtain similar results under the
assumption of $N\geq 2d$. We lower the measurement number $d+1$ (resp. $2d$)
with showing that there exist $N=d$ (resp. $2d-1$) matrices $A_1,\ldots, A_N\in
\mathbf{H}_d(\mathbb{R})$ (resp. $\mathbf{H}_d(\mathbb{C})$) which have the
almost everywhere phase retrieval property. Our results are an extension of
almost everywhere phase retrieval from the standard phase retrieval to the
general setting and the proofs are often based on some new ideas about
determinant variety.
