Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference
F. Wrede
R. Eriksson
R. Jiang
L. Petzold
A. Hellander
P. Singh
F. Wrede
R. Eriksson
R. Jiang
L. Petzold
A. Hellander
P. Singh
State-of-the-art neural network-based methods for learning summary statistics have delivered promising results for simulation-based likelihood-free parameter inference. Existing approaches for learning summarizing networks are based on deterministic neural networks, and do not take network prediction uncertainty into account. This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and produces a proposal posterior density using categorical distributions. An adaptive sampling scheme selects simulation locations to efficiently and iteratively refine the predictive proposal posterior of the network conditioned on observations. This allows for more efficient and robust convergence on comparatively large prior spaces. The approximated proposal posterior can then either be processed through a correction mechanism to arrive at the true posterior, or be used in conjunction with a density estimator to obtain the posterior. We demonstrate our approach on benchmark examples.% and compare against related methods. % Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space. % % ** 1 ** % Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account. % This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions. % %This work proposes an uncertainty-enhanced summary statistics learning framework using Bayesian neural networks, which incorporates epistemic uncertainties to quantify the confidence of the trained model. % % ** 2 ** % An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network. % %conditioned on observations, hence eliminating the need for discrepancy-based rejection or density estimation and reducing the number of simulations required. % % ** 3 ** % %Our novel approach also includes estimating the posterior using categorical distributions, subsequently transforming the learning task into a classification problem. % This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.
abstract.tex
\documentclass[a4paper]{article}
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{float}
\usepackage[font=small,labelfont=bf]{caption}
\usepackage[a4paper,top=1cm,bottom=2cm,left=4cm,right=4cm,marginparwidth=1.75cm]{geometry}
\begin{document}
%\title{Making Sense of Large Computational Experiments}
\title{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\date{}
%Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference
%Likelihood-free parameter inference using supervised sequential bayesian neural networks
%\author{F. Wrede, R. Eriksson, R. Jiang, L. Petzold, A. Hellander, P. Singh}
%\footnote{Presenting author.}
\maketitle
\baselineskip=0.9\normalbaselineskip
\vspace{-10pt}

State-of-the-art neural network-based methods for learning summary statistics have delivered promising results for simulation-based likelihood-free parameter inference. This work proposes an uncertainty-enhanced summary statistics learning framework using Bayesian neural networks, incorporating epistemic uncertainties to quantify the confidence of trained models. An adaptive sampling scheme selects simulation locations to efficiently and iteratively refine the predictive posterior of the network conditioned on observations, eliminating the need for discrepancy-based rejection and reducing the number of simulations required. Our novel approach includes estimating the posterior using categorical distributions, subsequently transforming the learning task into a classification problem, thus allowing for efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against similar state-of-the-art methods. 

\end{document}
NOTINUSEmain.tex
%%%%%%%% ICML 2021 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%%

\documentclass{article}

% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{booktabs} % for professional tables

% Added by authors
\usepackage{amsmath}
\usepackage{amssymb}

% hyperref makes hyperlinks in the resulting PDF.
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
% please comment out the following usepackage line and replace
% \usepackage{icml2021} with \usepackage[nohyperref]{icml2021} above.
\usepackage{hyperref}

% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Use the following line for the initial blind version submitted for review:
%\usepackage{icml2021}

% If accepted, instead use the following line for the camera-ready submission:
\usepackage[accepted]{icml2021}

% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{Robust and integrative Bayesian neural networks for likelihood-free parameter inference}

\begin{document}

\twocolumn[
%\icmltitle{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\icmltitle{Robust and integrative Bayesian neural networks\\ for likelihood-free parameter inference}
% It is OKAY to include author information, even for blind
% submissions: the style file will automatically remove it for you
% unless you've provided the [accepted] option to the icml2021
% package.

% List of affiliations: The first argument should be a (short)
% identifier you will use later to specify author affiliations
% Academic affiliations should list Department, University, City, Region, Country
% Industry affiliations should list Company, City, Region, Country

% You can specify symbols, otherwise they are numbered in order.
% Ideally, you should not use this facility. Affiliations will be numbered
% in order of appearance and this is the preferred way.
\icmlsetsymbol{equal}{*}

\begin{icmlauthorlist}
\icmlauthor{Fredrik Wrede}{equal,uu}
\icmlauthor{Robin Eriksson}{equal,uu}
\icmlauthor{Richard Jiang}{ucsb}
\icmlauthor{Linda Petzold}{ucsb}
\icmlauthor{Stefan Engblom}{uu}
\icmlauthor{Andreas Hellander}{uu}
\icmlauthor{Prashant Singh}{umu}
\end{icmlauthorlist}

\icmlaffiliation{uu}{Department of Information Technology, Uppsala University, Uppsala, Sweden}
\icmlaffiliation{ucsb}{Department of Computer Science, University of California-Santa Barbara, Santa Barbara, CA, USA}
\icmlaffiliation{umu}{Department of Computing Science, Umeå University, Umeå, Sweden}

\icmlcorrespondingauthor{Fredrik Wrede}{fredrik.wrede@it.uu.se}
\icmlcorrespondingauthor{Prashant Singh}{prs@cs.umu.se}

% You may provide any keywords that you
% find helpful for describing your paper; these are used to populate
% the "keywords" metadata in the PDF but will not be shown in the document
\icmlkeywords{Machine Learning, ICML}

\vskip 0.3in
]

% this must go after the closing bracket ] following \twocolumn[ ...

% This command actually creates the footnote in the first column
% listing the affiliations and the copyright notice.
% The command takes one argument, which is text to display at the start of the footnote.
% The \icmlEqualContribution command is standard text for equal contribution.
% Remove it (just {}) if you do not need this facility.

%\printAffiliationsAndNotice{}  % leave blank if no need to mention equal contribution
\printAffiliationsAndNotice{\icmlEqualContribution} % otherwise use the standard text.

\begin{abstract}
State-of-the-art neural network-based methods for learning summary statistics have delivered promising results for simulation-based likelihood-free parameter inference.
Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account. 
This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions. 
An adaptive sampling scheme selects simulation locations to efficiently and iteratively refine the predictive posterior of the network conditioned on observations.
This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.
% Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space.
% % ** 1 **
% Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account. 
% This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions.
% %This work proposes an uncertainty-enhanced summary statistics learning framework using Bayesian neural networks, which incorporates epistemic uncertainties to quantify the confidence of the trained model. 
% % ** 2 **
% An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network.
% %conditioned on observations, hence eliminating the need for discrepancy-based rejection or density estimation and reducing the number of simulations required. 
% % ** 3 **
% %Our novel approach also includes estimating the posterior using categorical distributions, subsequently transforming the learning task into a classification problem. 
% This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.
\end{abstract}

\section{Introduction}
\label{sec:intro}
 Likelihood-free inference (LFI) is a class of methods designed for parameter inference of models with intractable or expensive likelihoods. 
 %The type of models range from ... 
 Approximations to the posterior distribution conditioned on observed data are made with the aid of forward simulations of the model, thus also referred to as simulation-based inference \cite{Cranmer_frontierSBI}. Approximate Bayesian computation (ABC)~\cite{sisson2018handbook} is a popular method in this setting, and involves the use of a discrepancy measure to compare simulated data against observed data. A threshold on the discrepancy measure decides whether a proposed parameter can be accepted as a sample from the posterior. 
 
While this approach has shown to be robust, it has certain drawbacks. For many dynamical models, the simulation output is high-dimensional, 
and one uses summary statistics as a dimension reduction technique
 % avoid large approximation errors
 \cite{wood2010statistical,engblom2020bayesian}. Selecting informative summary statistics so that the discrepancy measure can accurately accept posterior samples is often a tedious task, typically requiring detailed domain knowledge and trial and error. 
 
 An alternative approach to handpicking summary statistics is to utilize embedded neural networks (NN) to learn summary statistics from simulated data \cite{jiang2017learning, dinev2018dynamic, wiqvist2019partially, aakesson2020convolutional,chen2020neural}.  Some of these methods require a post-processing ABC scheme to yield the posterior, others work in conjunction with neural density estimators \cite{lueckmann2021benchmarking} to either estimate the posterior directly, or use synthesized likelihoods which requires Markov chain Monte Carlo (MCMC) sampling. Although deterministic embedded networks for learning summary statistics have shown impressive results, they are prone to be inaccurate in regions where the training data is sparse or not well represented (particularly in the case of problems with high-data dimensionality). 
 %The uncertainty of prediction or network prediction confidence is not taken into account in the LFI pipelines.
 %they can often lead to too `overconfident' predictions\textcolor{red}{cite}. Although implementations of embedding networks include regularization techniques to avoid overfitting, they still suffer from inaccuracies in regions of the parameter space not well-represented in the training data. 
Deterministic NNs do not directly provide an estimate of their prediction uncertainty, which can potentially mislead the parameter inference process.
%Deterministic NNs cannot quantify uncertainties associated with unseen data independently, which manifests as an inherent problem when one samples a finite amount of parameter proposals from large, potentially uninformative priors. 
This lack of direct uncertainty quantification motivates the design of more robust and expressive embedding networks, that can drive the LFI process in a more reliable manner.
 
 % ToDo for PS: elaborate more on the overconfident part - this is due to (limited) sampling in hd spaces, with large gaps of no data
 % PS: Perhaps we should clearly say - Approach 1: (BNN) and Approach 2: (BNN + classification)
This paper presents a novel likelihood-free parameter inference framework that improves upon two characteristics of embedded NN-based LFI methods, as outlined below.
\begin{itemize}
    \item Bayesian embedded NNs are proposed as summary statistics that provide prediction uncertainties, and also directly approximate the posterior. An adaptive sampling algorithm utilizes these uncertainty estimates to iteratively refine the approximated posterior. Including prediction uncertainty in the LFI process provides robustness against potential gaps in training data.
    \item As the Bayesian embedded NN directly models the posterior distribution, no separate post-processing (e.g., density estimation or MCMC) is needed. 
\end{itemize}
 
%This work explores Bayesian embedded networks as trainable summary statistics that provide prediction uncertainties, and directly estimate the posterior distribution.
%Introducing epistemic uncertainties in the embedded network will enable a more robust LFI workflow by considering model confidence in addition to model predictions. The embedded network's predictive posterior will directly estimate the posterior distribution for the inference task, ultimately integrating embedded summary statistics networks with neural density estimators. 
The proposed approach involves approximating the posterior through discrete categorical distributions, subsequently solving a regression problem via classification. This approach in similar form has delivered encouraging results~\cite{weiss1995rule, torgo1996regression, AHMAD_RvC}. The main motivation behind this design is that performing regression with high-dimensional inputs (such as dynamical simulation responses) to continuous multi-dimensional outputs (such as parameters) is a highly challenging task. Even though NNs have the learning capacity to do so, generalizing the networks often proves difficult. 
%Particularly if the training data is not fixed over individual training procedures, such as the case for parameter inference. It's our intention to make parameter inference using NNs more robust by utilizing regression via classification.     

 We demonstrate the efficiency and robustness of our approach on benchmark problems and compare against the deterministic counterparts in conjunction with neural density estimators. Experimental results show  that the proposed approach is consistently more robust than existing methods, with relatively small variance in parameter inference quality performance. The error in parameter estimation is comparable to existing methods. %\textcolor{red}{Think more about how we add value over SNPE}
 %As an alternative approach, some methods have been focusing on introducing inverse mapping functions $f$.  By fitting the inverse mapping function $f(\bold{y}, w) \mapsto \hat{\theta}$ using simulated data  $X = \{y_i, \theta_i\}^N$, a loss function $L(\theta_i, \hat{\theta_i})$ can be optimized w.r.t $w$ between predicted and true parameters, the weights $w$ should ultimately incorporate the discrepancy between samples. While some variants can directly yield density estimators $f(y_0,w) \mapsto \hat{p}(\theta|y_0}$, others use it in a post-processing rejection scheme by letting $d(f(y_i,w), f(y_0,w))$. Learning the inverse mapping function is typically referred as a regression problem where one of the most popular approaches is to use artificial neural networks (ANN).}

%However, most applied deep learning architectures are deterministic and non-probabilistic and can yield overconfident predictions on targets. Further, classification as shown to be superior to regression in prediction performance when $f$ inherits complex non-linear patterns. Thereof, we explore using Bayesian neural networks to better account for epistemic uncertainty and regularization in the mapping function and we discretize $\theta$ space into categorical classes to transform the problem into a classification task, assuming that sample paths $y$ to be similar when in parameter proximity. The inherent properties of dynamical processes makes convolutional neural networks (CNN) an attractive option for learning the mapping function $f$. CNNs make use of a filtering kernel to create local feature maps.Previous studies have demonstrated the capabilities of using determinstic CNNs for likelihood-free parameter inference \cite{}. In this study we use a similar architecture but make use of probablistic layers in Tensflow probability package to place parameterized distributions over the weights of the neural network and apply variational inference using evidence lower bound (ELBO) and \textit{Bayes by Backprop} to fit a posterior distribution. This approach suggests that, not one, but an ensemble of neural networks are learned, where each network's weights are sampled from the fitted posterior. While it might seem computationally heavy learning an infinite ensemble of networks, in fact the amount of weights only increases by a factor of two when using Monto Carlo estimates of the gradients on an otherwise intractable problem.
We organize the paper as follows: Section~\ref{sec:background} introduces the problem setting, notation, and related work. Section~\ref{sec:method} presents the proposed LFI parameter estimation approach. Section~\ref{sec:experiments} demonstrates the proposed methodology on benchmark examples and draws comparisons to other methods. Section~\ref{sec:discussion} presents a discussion on the merits and applicability of the approach. Section~\ref{sec:conclusion} finally gives concluding remarks about the novel method presented in the paper.

\section{Background} %  notation, estimate PDF with Neural net (with embedding net)
\label{sec:background}
% problem set-up | we want the posterior given model M, and observation y0
Let ${\bf y}_0$ be a target observation which can be approximately modeled by $M({\bf y} | {\boldsymbol \theta})$.
%Assume a dynamical model  $\mathcal{M}$ which has generated an observation ${\bf y}_0$. Next, assume some model error by approximating the model by $M({\bf y} | {\boldsymbol \theta})$, with parameters $\boldsymbol \theta$.
Forward simulation of the model $M$ results in the sample path ${\bf y}^{1:T}$ henceforth referred to as ${\bf y}$.
%Let $ {\bf y}_0 \sim M({\bf y}| {\boldsymbol \theta})$ be observed data approximately sampled from $M({\bf y}|\boldsymbol{\theta})$, i.e., we assume some model error.
%The observed data may also have been obtained from the physical process corresponding to $M$. 
The objective is to infer the parameters conditioned on the observation, i.e., the posterior density $p(\boldsymbol{\theta} | {\bf y}_0) \propto p({\bf y}_0 | \boldsymbol{\theta})p(\boldsymbol{\theta})$. However, as we assume that the likelihood is computationally intractable, which is often the case for dynamical models, an estimate $\hat{p}(\boldsymbol{\theta} | {\bf y}_0)$ of the true posterior distribution must be approximated. Approaches in this setting typically utilize access to the observed data  ${\bf y}_0$ and the ability to perform simulations from model $M$ at well-chosen parameter locations. We summarize some the methodologies under the supposed setting below.
%, and motivate the need for a fully Bayesian model-driven LFI approach.


 
 

\subsection{Related work}
% --- Simulation based Inference --- %
% discrepancy-, SS based
The ABC family of methods forms the most popular and established LFI approach (introduced in Section~\ref{sec:intro}). 
%The methodology consists of a sample-simulate-compare cycle where parameter samples $\boldsymbol{\theta}_i \sim p(\boldsymbol{\theta})$ drawn from a defined prior distribution $p(\boldsymbol{\theta})$ are simulated using the model $M$ to obtain sample paths ${\bf y}$, which one consequently compare to observed data ${\bf y}_0$ to either declare T1 as being accepted as part of the estimated posterior or rejected. The workflow consists of several hyperparameters, including the choice of the prior P1, dimension reducing summary statistics (when ${\bf y}$ is high dimensional, e.g., time series), a distance or discrepancy measure to compare simulated and observed data, and a tolerance bound $\epsilon$ to define acceptance/rejection. 
Although remarkably successful, ABC methods are typically very challenging to setup optimally for accurate inference quality and convergence speed \cite{sisson2018handbook,marin2012approximate}. Extensions such as sequential Monte Carlo methods (ABC-SMC) \cite{sisson2007sequential} can potentially accelerate the convergence speed but still require laborious hyperparameter selection. In particular, the choice of informative summary statistics is critical for practical and accurate parameter inference as it forms the means of comparing simulated proposals with the observation. 

Consequently, summary statistic selection has received considerable attention \cite{sisson2018handbook}[Chapter 5]. However, traditional summary statistics such as statistical moments may not always be expressive enough to capture intricate details in rich dynamical data. Further, ideal statistics may even be missing from the candidate pool of statistics to select from. A recent development has been the use of data-driven regression models to extract features from data, and function as summary statistics. It has been shown that the ideal summary statistic is the posterior mean, when minimizing the quadratic loss \cite{fearnhead2012constructing}. In particular, NN based regression models have delivered highly promising results towards learning the estimated posterior mean as summary statistics from high-dimensional detailed time series \cite{jiang2017learning,wiqvist2019partially,aakesson2020convolutional}. Such models learn the mapping $F_{\bf w}({\bf y}) \mapsto \mathbb{E}[\boldsymbol{\theta}]$ which can be seen as an amortized point estimate for a given ${\bf y}$, where ${\bf w}$ are the weights of the regression model. However, an overhead associated with \emph{trainable} summary statistics is the need for large amounts of simulated training data to yield an accurate  posterior mean estimate. Although there are opportunities for re-using the training data in the later rejection sampling step to estimate the conditioned posterior, the sheer amount of training data required is problematic whenever the simulations are expensive to compute. % Mention that they can't quantify uncertainties about their estimates 

%Synthetic likelihood based approaches \cite{wood2010statistical}  step away from discrepancy distances and approximate the likelihood conditioned on the summary statistics.

% remove SS, still discrepancy-dependent
%Efforts to use embedding nets to compute optimal summarizing statistics have shown great promise, NN \cite{jiang2017learning}, Convolutional NN \cite{aakesson2020convolutional}, but they still necessitate a discrepancy distance to evaluate proposals. These approaches consists of fitting a regression model $F_w(y) \mapsto \theta$ parameterized by the weights $w$ and subsequently use prediction on simulated data
%\textcolor{red}{FW: need to introduce SNPE-C in particular\cite{greenberg2019automatic}}
Neural density estimators form a family of methods that directly estimate densities with the aid of NNs and have shown impressive performance on challenging problems \cite{lueckmann2021benchmarking}, such as the masked autoregressive flows (MAF) and the neural spline flow (NSF) \cite{papamakarios2017masked,NSF}. While some neural density estimators are used to estimate posterior densities \cite{APT_2019,SNPE_A_Papamakarios, SNPE_B_lueckmann2017flexible}, others rely on constructing synthesized likelihoods or ratios which require MCMC sampling to extract the posterior \cite{papamakarios2019sequential, lueckmann2019likelihood,SNRE_a, SNRE_b_constrative}. The formed posteriors are amortized, i.e., any observation can be conditioned on the posterior. 

However, a collection of algorithms based on neural density estimators uses sequential Bayesian updates to the posterior, conditioned on a particular target observation. These include sequential neural posterior estimation (SNPE) \cite{SNPE_A_Papamakarios, SNPE_B_lueckmann2017flexible, APT_2019},  sequential neural likelihood estimation (SNLE) \cite{papamakarios2019sequential, lueckmann2019likelihood} and sequential neural ratio estimation (SNRE) \cite{SNRE_a, SNRE_b_constrative}. A benchmarking effort was recently conducted comparing the performance of these approaches on various problems \cite{lueckmann2021benchmarking}. These approaches also support the use of embedding networks to turn high-dimensional simulation outputs to informative summary statistics. The weights of the embedding networks are then trained together with the weights of the neural density estimator. Thus, these shared architectures enable sequential updates on the conditioned posterior and the summary statistics learning process, ultimately reducing the amount of simulations required. 

SNPE-C (also known as automatic posterior transform - APT) \cite{APT_2019} is a recent SNPE technique that learns to infer the true posterior by maximizing an intermediate estimated proposal posterior. SNPE approaches form a good class of methods to compare the proposed approach to, as the use of sequential Bayesian updates to refine the estimated posterior is close in spirit to adaptive sampling used in this work. SNPE approaches have also been shown to perform well across a variety of test problems \cite{lueckmann2021benchmarking,APT_2019}. Among SNPE approaches, SNPE-C is superior as it is more flexible and scalable as compared to SNPE-A and SNPE-B \cite{APT_2019}, and is used for the purpose of comparisons in the experiments in this paper. 


%The most recent and superior version of SNPE is the Automatic Posterior Transform (APT) \cite{APT_2019}, referred as SNPE-C in the software package SBI\footnote{https://www.mackelab.org/sbi/} which is the naming we will be using henceforth.       
    
% A list, that then has to be made into a flowing text.
% *classification*
%Using classification in LFI is not novel~\cite{gutmann2018likelihood}.  
%
% *flow-based*
%Flow-based model $q(\eta|x)$ condition on $x$ to generate the next training data set $\mathcal{D}$ in order to approximate the posterior $p(\eta|x)$, APT (SNPE-C) \cite{greenberg2019automatic, gonccalves2020training}
%Flow-based model with $q(\eta|x)$ conditioned on $\eta$ \cite{papamakarios2019sequential}
%SNPE \cite{lueckmann2017flexible,lueckmann2019likelihood,papamakarios2017masked}
% --- collecting ---
%Benchmarking effort of SBI \cite{lueckmann2021benchmarking}, 

\subsection{Motivation for Bayesian neural networks}
We consider a different approach to neural density estimation. We implement a shared, embedded network for summary statistics, with density estimation through Bayesian neural networks (BNNs). The introduction of weight uncertainties in NNs has enabled a robust means for learning an ensemble of models \cite{Bayesbybackprop}, represented by the posterior density $p({\bf w}|{\bf y})$ where ${\bf w}$ are the network weights.
%Blundell \textit{et al.} demonstrate with their Bayes by backprop algorithm how one can obtain and fit an ensemble of models represented by the posterior density $p(w|X)$ where $w$ is the network weights and $X$ the training data. 
It has also been demonstrated \cite{Bayesbybackprop} that the number of weights only increases by a factor of two by utilizing Monto Carlo (MC) sampling of gradients when updating the weights. The benefit of using a BNN is that the posterior predictive distribution $p(\boldsymbol{\theta}|{\bf y}) = \int p(\boldsymbol{\theta}|{\bf w}, {\bf y})p({\bf w}|{\bf y})d{\bf w}$ provides an uncertainty measure to its predictions. The uncertainty estimates form an elegant means of identifying and countering overfitting, which translates into more robust and efficient embedding networks. Different embedding network architectures such as RNNs or CNNs can be implemented in the BNN setting to provide prediction uncertainty estimates, and directly estimate the posterior $p(\boldsymbol{\theta}|{\bf y}_0) \approx p(\boldsymbol{\theta}|{\bf y})$ by the posterior predictive distribution. The uncertainty estimates also allow sequential adaptive sampling of the posterior, conditioned on a target observation for faster convergence, similar to SNPE. 

%However, due to the integral in the predictive posterior being analytically intractable, numerical integration needs to be utilized (e.g., using MC methods). For the application of LFI, a BNN will provide the flexibility of using embedding networks, e.g., architectures such as RNNs or CNNs, that provide uncertainty estimates of predictions, and directly estimate the posterior $p(\theta|y_0) \approx p(\theta|X)$ by the posterior predictive distribution. This, in turn will enable sequential adaptive sampling of the posterior conditioned on a target observation for faster convergence. 
%\textcolor{red}{FW: Not completely sure of the notation of posterior predictive distribution in our case is it $p(\theta|X)$, $p(y_0|X)$, $p(\theta|w,X)$ etc??} 
%\textcolor{blue}{PS: $p(\theta|X)$, or $p(\theta|w,X)$ (there are arguments on both sides whether or not to include $w$. I'd go with $p(\theta|X)$. It is okay to abstract model specifics (in the form of $w$).}
%The need for large quantities of (potentially computationally expensive) simulated data for training summary statistics and subsequently estimating the posterior distribution using the ABC family of methods can lead to impractical parameter inference processing times. Therefore, this paper aims at developing a data-efficient LFI parameter estimation pipeline that minimizes the training data needed to obtain accurate high-quality summary statistics using an adaptive sampling algorithm, and directly estimates the posterior using Bayesian neural networks thereby eliminating the need for ABC-based posterior estimation. Further, an efficient classification-based approach is also presented that iteratively shrinks the prior search space.
% 
% 

\section{Likelihood-free inference via Bayesian neural networks}
\label{sec:method}
 Let $D = \{({\bf y}_i,\boldsymbol{\theta}_i)\}_1^N$ consist of simulated data and parameter proposals sampled from the prior distribution over the parameter space $p(\boldsymbol{\theta})$. Consider a BNN, parameterized by the weights ${\bf w}$, that computes the posterior density $p({\bf w}|{\bf y})$ of a probabilistic model $p(\boldsymbol{\theta}|{\bf y},{\bf w})$ by placing a prior $p({\bf w})$ on the weights. The posterior predictive distribution for new unseen sample path ${\bf y}_t$,  $p(\hat{\boldsymbol{\theta}}|{\bf y}_t) = \mathbb{E}_{p({\bf w}|D)}[p(\hat{\boldsymbol{\theta}}|{\bf y}_t, {\bf w})]$ is the expectation on unseen data. Setting ${\bf y}_t = {\bf y}_0$, for $\hat{\boldsymbol{\theta}} \approx \boldsymbol{\theta}$ the posterior predictive distribution will be equal to $p(\boldsymbol{\theta}|{\bf y}_0)$ as $N \rightarrow \infty$, which is the target parameter posterior conditioned on a particular observation. Additionally, when the probabilistic model $p(\boldsymbol{\theta}|{\bf y},{\bf w})$ is a regression model, it also will yield $\hat{\boldsymbol{\theta}} = \boldsymbol{\theta}$. However, as we demonstrate next, we will approximate the parameters $\boldsymbol{\theta}$ through discrete categories making $p(\boldsymbol{\theta}|{\bf y},{\bf w})$ a categorical distribution thereby transforming the regression problem into a classification problem. 

\subsection{Probabilistic regression via classification}
Let $p(\boldsymbol{\theta}'|{\bf y},{\bf w})$ be represented as a categorical distribution using cross-entropy and softmax activation in a NN classifier denoted by $F_{p(\boldsymbol{\theta}'|{\bf y},{\bf w})}$.  We will transform the continuous problem by discretizing $\boldsymbol{\theta} \in \mathbb{R}$ into $\boldsymbol{\theta}'\in K$ by $B$ equal width bins per dimension (please see Section~\ref{sec:discussion} for a discussion on binning strategies), i.e., the predictive distribution will provide the probability of belonging to a particular bin $K \in {1,2,...,B}$. By placing a multivariate Gaussian on $p({\bf w})$ we will fit $p({\bf w}|D)$ using Bayes by backprop, subsequently yielding the BNN classifier  $F_{p({\bf w}|D)}$. To estimate the posterior predictive distribution conditioned on our target observation, we sample from $F_{p({\bf w}|D)}$ using a MC sampling scheme,   
\begin{equation}
    p(\boldsymbol{\theta}'|{\bf y}_0) \approx \frac{1}{N}\sum_{i=1}^N F_{p({\bf w}|D)}({\bf y}_0, {\bf w}_i).
\end{equation}

To enable sequential adaptive sampling during the inference task, we will train $G$ subsequent classifiers $F^G_{p({\bf w}|D)}$ where the bins are updated by sampling from the posterior predictive distribution. Each iteration employs the density from the previous generation as the prior for sampling new training data. The BNN generates amortized posterior estimates. However, when conditioned on the observation, it allows for adaptive refinement in each iteration. To be able to simulate new samples, we need to transform $\boldsymbol{\theta}'$ back to $\boldsymbol{\theta}$. This is achieved by transforming each bin $K$ into a uniform distribution,

\begin{equation}
\begin{aligned}
    \{\boldsymbol{\theta}'\}_{i=1}^N &\sim p(\boldsymbol{\theta}'|{\bf y}_0) \\
    \{\boldsymbol{\theta}\}_{i=1}^{N} &= \{\mathbb{U}^K({\boldsymbol{\theta}'}_i^{\text{\emph{left}}},{\boldsymbol{\theta}'}_i^{\text{\emph{right}}})\}_{i=1}^N,
\end{aligned}
\end{equation}

where \emph{left} and \emph{right} are the boundaries of the bin intervals. The adaptive sampling-driven BNN is depicted in Figure~\ref{fig:pipeline} and detailed in Algorithm~\ref{alg:posterior}.

\begin{figure}[t]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/pipeline_v3.pdf}}
\caption{Adaptive sampling-driven BNNs. (A) Parameter proposals are sampled from a prior and simulated via the model. (B) Regression via classification. Continuous proposals are discretized into a categorical distribution with classes being the bins. A BNN (having prior weights) is then trained on data. (C) The posterior predictive conditioned on observation $y_0$ can be estimated via MC sampling, and is subsequently used in an iterative procedure, replacing the previous prior. Note that a transformation is applied to the categorical distribution to transform proposals back to continuous.}
\label{fig:pipeline}
\end{center}
\vskip 0.2in
\end{figure}

\subsection{Exploitation of bins}
%With large priors $p(\boldsymbol{\theta})$, finding the subset $p(\boldsymbol{\theta}|{\bf y}_0)$ can be a slow process.% 
We aim to exploit the uncertainty estimates provided by the posterior predictive distribution to enable faster convergence. A key reason why the process can be slow is that low probability weights linger between iteration updates. We exploit the posterior predictive distribution by using a threshold value, $\delta$. Given the categorical distribution, we set the classes with probability weight below the threshold to zero. The non-zero probabilities are then normalized, constructing a less skewed proposal distribution. Between iterations, $\delta_i$ can be chosen adaptively or kept constant throughout. We recognize that the sequence of distinct $\delta_i$ holds a similarity to the discrepancy thresholds ($\epsilon_i$) used by ABC-SMC~\cite{sisson2007sequential, del2012adaptive}.
% ** if not adaptive **
The potential adaptivity is briefly discussed in Section~\ref{sec:discussion}.
%% *** if we decide on using adaptive ***
%To slow the exploitation, we employ an $\epsilon$ that converges to zero as the rounds increase.



\subsection{Dealing with the curse of dimensionality}
For the single-parameter problem, the structure of the classification problem is straightforward: one classifier for one parameter. However, for multiple parameter dimensionality, we construct one classifier for each two-dimensional covariance combination: $(\boldsymbol{\theta}_i, \boldsymbol{\theta}_j)$, implying that for a model of $k$ parameters, we would require $\frac{k!}{(k-2)!2!}$ classifiers. One sees that the number of classifiers quickly increases; however, they lend themselves towards an embarrassingly parallel workflow since they are independent. Further, the adaptation of the proposals is done per covariance dimension and then marginalized, resulting in the $k$-dimensional parameter proposals.


%The LFI problem is solved by finding the inverse map $F(y)\mapsto \theta$. Consider the approximation $F_w(y)$ produced by an embedded net with weights, $w$. By fitting the inverse mapping function $F_w(y)$ using simulated data $\mathcal{D} = \{(y_i,\theta_i)\}_1^N$, a loss function $\mathcal{L}(\theta_i, \hat{\theta_i})$ can be optimized w.r.t. $w$ between predicted and true parameters; the weights $w$ should ultimately incorporate the discrepancy between samples.
%Our probabilistic approach allows incorporating epistemic uncertainty by employing Bayesian embedding nets (BNN) \cite{mackay1992practical}, which learn distributions of $q(w|\mathcal{D})$ instead of a point estimate $\hat{\theta}$ by introducing a prior $p(w)$ on the weights. Approximating the predictive posterior $p(y_0 |\mathcal{D})$ by Monte Carlo integration gives an interpretation of $\hat{p}(\theta | y_0)$, namely approximating the integral $ p(y_0 |\mathcal{D}))  = \int p(y_0 |\mathcal{D}),w)q(w|\mathcal{D}))dw$
%by a sum of independent samples $ p(y_0 |\mathcal{D})) \approx \frac{1}{M}\sum_{i=1}^M p(y_0 | w_i)$
%where $w_i \sim q(w | \mathcal{D}))$. 
%A connection between the NN and the BNN is that NN's weights can be seen as maximum likelihood estimates or maximum a posteriori estimates, when using regularization. %\cite{}? could use, https://arxiv.org/pdf/2007.06823.pdf but not sure how close they are to be published results.

%\begin{equation}
%    \begin{aligned}
%        p(y_0 |\mathcal{D}))  = \int p(y_0 |\mathcal{D}),w)q(w|\mathcal{D}))dw\\
%         \approx \frac{1}{M}\sum_{i=1}^M p(y_0 | w_i)\\
%         \approx \hat{p}(\theta|y_0)\\
%         \text{where } w_i \sim q(w | \mathcal{D}))
%    \end{aligned}
%\end{equation}


% The classification problem
%\subsection{Accelerating posterior estimation using classification}
%To achieve robust results (\textcolor{red}{FW: this is not enough motivation for using classification}), around the sought after posterior distribution $\hat{p}(\theta|y_0)$, we consider a sequence of discrete mappings from $y \mapsto \theta$ with increasing resolutions. Therefore, we solve finding the continuous map $F$ around $y_0$ by a series of approximate mappings, i.e., $F = f_1 \circ f_2 \circ \dots \circ f_G$. 
%The discrete map $f_k$ considers binned continuous data, meaning, finding $f_k$ is a classification problem for a categorical distribution. We condition each iterative map on $y_0$ to determine the area of interest in the prior, i.e., $f_{k+1}$ follows $f_k$, and $P_{f_k}(\theta|Y_0)$ gives the samples of interest. 
% The final inference step
%A key feature inherent of BNN is that the map $F$ direcred}tly gives the probability $p(\theta|y)$, and given the observation $y_0$, we get $\hat{p}(\theta|y_0)$ without any further steps.


%\section{Classification for LFI using Bayesian neural networks}
%Our approach's contribution is that the BNN allows for uncertainty estimates directly while on the classification step, instead of using the accuracy as a pseudo-measure for the discrepancy between the observation and simulation. We construct the classes by grouping continuous-data into a fixed number of bins. As the number of bins approach the number of samples, we approach the continuous, regression case.  

%The posterior density estimation by BNN is generated iteratively by a sequence of generation densities.  Each iteration employs the previous generation density as it's prior for sampling new training data. The BNN generates amortized posterior estimates. However, when conditioned on the observation, it allows for adaptive refinement in each iteration.

%Denote the trained BNN as $F(y|w)$ with weights $w$, the posterior distribution, approximated by the BNN, $q_{F(y,w)}$. We generate $q_F$ by Monte Carlo-sampling the network, and retrieve a categorical distribution over the binned data.


\begin{algorithm}[]
   \caption{Iterative BNN posterior estimation }
   \label{alg:posterior}
\begin{algorithmic}
   \STATE {\bfseries Input:} model simulator $M({\bf y}|\boldsymbol{\theta})$, 
   observations ${\bf y}_0$, Bayesian neural network $F_{p({\bf w}|D)}$, 
   number of generations $G$, number of simulations per generation $N$,
   prior distribution $p(\boldsymbol{\theta})$, number of bins $B$.

   \STATE $p_0 := p$  
   \FOR{$g=1$ {\bfseries to} $G$}
        %\FOR{$n=1$ {\bfseries to} $N$} % training
            \STATE Sample $\boldsymbol{\theta}_N = \{\boldsymbol{\theta}_i\}_{i=1}^N$ from $p_{g-1}(\boldsymbol{\theta})$
            \STATE Sample ${\bf y}_N = \{{\bf y}_i\}_{i=1}^N$ from $M({\bf y}|\boldsymbol{\theta}_i)$
       % \ENDFOR
        \STATE Group $\boldsymbol{\theta}_N$ into K bins $\boldsymbol{\theta}'_N, \, K= 1:B$
        \STATE Train $F^g_{p({\bf w}|D_g)}$ on $D_g = \{{\bf y}_N, \boldsymbol{\theta}'_N\}$
        \STATE Estimate $p_{g}(\boldsymbol{\theta}'_N|{\bf y}_0) \leftarrow \mathbb{E}[F^g_{p({\bf w}|D_g)}({\bf y}_0,{\bf w}_N)]$
        \STATE Transform $p_g(\boldsymbol{\theta}) \leftarrow \mathbb{U}^K({p}_{g}(\boldsymbol{\theta}'_N|{\bf y}_0))$
        %\STATE \textcolor{red}{FW: Not sure what notation to use for the transform}
    \ENDFOR
    \STATE {\bfseries return} $p_g(\boldsymbol{\theta})$
\end{algorithmic}
\end{algorithm}

%\subsection{Complexity}
%\textcolor{red}{FW: Can we say something more about the complexity of the approach more than the number of weights? E.g simulations $>>$ training time? Also curse of dim, several classifiers are embarrassingly parallel and the number of simulations does not increase. Very important that the reader understand this.} 

%\subsection{Design of the BNN}
%\textcolor{red}{FW: I think this part should go into experiments as a particular case of the BNN tailored for dynamic models}
%In this work we have implemented the BNN as Convolutional NN, henceforth called BCNN. However, we would to point out that many other types of architectures can be used. Out motivation behind this choice is that CNNs have shown great performace in deducting inormation from time-series data \cite{sadouk2019cnn, aakesson2020convolutional}, and not only for images. To turn the layers of a CNN into probabilitic layes we have used the flipout \cite{wen2018flipout} layes supported in Tensorflow \cite{tensorflow2015-whitepaper} probability package. 

%\subsection{Adaptive sampling}


\section{Experiments}
\label{sec:experiments}
%\textcolor{red}{FW: missing transition into CNN (BCNN), have not decided if it should be introduced in section \ref{sec:method} or as a particular case of a BNN in section \ref{sec:experiments}}

To test our approach we have implemented a convolutional BNN (we denoted this particular implementation BCNN). However, we note that our approach is not restricted to these types of architectures.  Our motivation behind choosing a convolutional architecture is that they have demonstrated excellent performance in identifying intricate patterns in complex, high-dimensional time-series data \cite{dinev2018dynamic, sadouk2019cnn, aakesson2020convolutional} as well as images~\cite{APT_2019}. For further information on the implementation we refer the reader to supplementary materials.  


We explore the performance of the BCNN on two benchmark examples - the moving averages model (MA) and the discrete-state continuous-time Markov chain implementation of the Lotka-Volterra (LV) \cite{lotka}. Together with the resulting posterior we recover using the BCNN, we also compare the inference performance to ABC-SMC and SNPE-C with the MAF as the neural density estimator.

The SBI package \cite{tejero2020sbi} implements SNPE-C, and is used for the experiments in this paper. To keep the comparison fair between SNPE-C and the BCNN, we designed the embedded nets to be as similar as possible between the two approaches, resulting in having approximately the same number of weights in the two models.  
 
All of the experiments were conducted on a machine consisting of a 12-core Intel Xeon E5-1650 v3 processor operating at 3.5 GHz, 64 GB RAM, running Fedora core 32 Linux operating system. Even though GPU-acceleration would speed up NN training substantially, in the considered setting we assume that simulations are significantly more expensive than training time. This is the case for the discrete and stochastic LV model, less so for MA.   


\subsection{Moving average model}%- Noisy, high data dimensionality
% \begin{equation}
%     X_t = \epsilon_t + \theta_1\epsilon_{t-1} + \theta_2\epsilon_{t-2},
%     \epsilon_t \sim N(0,1) 
% \end{equation}
The moving average (MA) model is a popular benchmark problem in the ABC setting \cite{marin2012approximate}. The MA model of order $q$, MA($q$) is defined for observations $X_1, \dots, X_p$ as \cite{wiqvist2019partially,aakesson2020convolutional},
\begin{equation}
    X_j = Z_j + \theta_1 Z_{j-1} + \theta_2 Z_{j-2} + \dots + \theta_q Z_{j-q},
\end{equation}
where $j=1,\dots,p$, and $Z_j$ represents latent white noise. For the experiments in this paper, we consider $q=2$ and $Z_j \sim N(0,1)$ \cite{wiqvist2019partially}. The MA(2) model is identifiable in the following triangular region,
\begin{equation}
    \theta_1 \in [-2,2], \theta_2 \in [-1, 1], \theta_2 \pm \theta_1 \geq -1.
\end{equation}
In order to ease implementation and compatibility between the compared approaches, a rectangular uniform prior encompassing the triangular region was used in the experiments. We note here that the estimated posterior was always within the triangular region, therefore no identifiability concerns were encountered due to a rectangular prior.

%\textcolor{red}{TODO: However we use a uniform prior...how do we motivate this choice? Both SNPE-C and ABC-SMC need a pdf of the prior. BCNN don't. Further the SBI package don't have an API supporting user-defined priors, which is baaaad}

\subsubsection{Experimental Setup}
In the experiment we conduct a total of $G=6$ iterations (rounds) of refining the posterior estimate for 10 individual seeds. The total amount of parameter proposals in each round is set to $N=3000$ and the number of bins for BCNN is set to $B=4$ per parameter dimension, resulting in a total of 16 bins. The exploitation threshold is set to $\delta = 0.05$. We simulate the MA(2) model with 100 equidistant time points.
For the ABC-SMC method, we use auto-correlation coefficients of lag 1 and 2, and variance as summary statistics. They are non-sufficient, but are chosen as a representation of a typical choice made by a practitioner in absence of much prior knowledge. Euclidean distance was used as the discrepancy measure.

\subsubsection{Results}
Here we demonstrate the perfomance of the proposed BCNN approach and compare against SNPE-C and ABC-SMC. The model has a superficial posterior, but the map $\boldsymbol{\theta} \mapsto {\bf y}$ is ill-discriminated, i.e., considerable variation in $\boldsymbol \theta$ provides little feedback in ${\bf y}$.  %Figure~\ref{fig:grid_MA2} A shows a qualitative evaluation of two sample executions(seed) of BCNN and SNPE-C. We observed that the BCNN is more robust in estimating the posterior as compared to SNPE-C, which we further demonstrate in Figure~\ref{fig:emd_MA2} by calculating the Earth Moving Distance (EMD, also referred as Wasserstein distance) and Maximum Mean Discrapancy (MMD) between estimated posteriors and the exact posterior. Finally, we also include ABC-SMC for reference; however, this method requires a substantial increase of simulations per round for convergence.  

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_grid_4_25x4_25_fontsize8.pdf}}
\caption{MA(2) per round posterior estimation for BCNN, SNPE-C, and ABC-SMC, together with the ground truth - a MCMC generated posterior. Each round includes $N=3000$ parameter proposals, except for ABC-SMC which require many more. BCNN and SNPE-C includes two different random seeds (1) and (2). Final column to the right shows a Gaussian kernel density estimate of the final round posterior per method.}
\label{fig:grid_MA2}
\end{center}
\vskip 0.2in
\end{figure}

Figure~\ref{fig:grid_MA2} provides qualitative insights on the shapes of the estimated posterior distributions using the different approaches. Figure~\ref{fig:emd_MA2} depicts the log mean earth mover distance (EMD)~\cite{levina2001earth} and the mean of maximum mean discrepancy (MMD) \cite{Scholkopf_MMD} between the estimated posteriors and the exact posterior. We see similar convergence patterns between BCNN and SNPE-C for both the EMD and MMD over subsequent rounds and different seeds, with the BCNN having a slight edge. It can also be observed that the BCNN is highly robust over different seeds with very little variance in performance.
Another interesting observation from Figure~\ref{fig:grid_MA2} is the improvement or refinement extracted by each approach over subsequent rounds. It can be seen that BCNN(1) and BCNN(2) are able to substantially improve the quality of the estimated posterior between rounds 2-4 and 4-6. The ability of the BCNN architecture to greatly improve over successive rounds can be attributed to the combination of uncertainty estimates from the BCNN model leveraged by the adaptive sampling algorithm to identify regions most likely to contain the true posterior.

%The lack of variability between seeds, the error bars in the figure showcase the robust properties of the BCNN method.

\begin{figure}[ht]

\includegraphics[width=\columnwidth]{img/ma2_emd_mmd_4.25x2_fontscale08.pdf}
\caption{Posterior error estimates, log mean EMD (left) and Mean MMD (right) for BCNN, SNPE-C and ABC-SMC per round, with variability $\pm$ 1 std, recovered from ten independent seeds. The BCNN have a slight edge over SNPE-C, especially when it comes to the variability. Note that on average ABC-SMC will require more simulations to converge than the other two, see bottom row in Figure~\ref{fig:grid_MA2}.}
\label{fig:emd_MA2}

\end{figure}

\subsection{Lotka-Volterra predator-prey model}
The Lotka-Volterra predator-prey dynamics model is a popular parameter estimation benchmark problem \cite{aakesson2020convolutional, greenberg2019automatic}. The model is implemented as a stochastic Markov jump process \cite{prangle2017adapting} simulated using the stochastic simulation algorithm (SSA) \cite{gillespie1977exact}. The model consists of three events - prey reproduction, predation and predator death. The model is described by the events:% $\mathcal{X}_1 \longrightarrow 2\mathcal{X}_1$, $ \mathcal{X}_1 + \mathcal{X}_2 \longrightarrow 2\mathcal{X}_2$ and $ \mathcal{X}_2 \longrightarrow \phi$.
\begin{equation}
 \begin{aligned}
     \mathcal{X}_1 &\xrightarrow{\theta_1} 2\mathcal{X}_1,\\
     \mathcal{X}_1 + \mathcal{X}_2 &\xrightarrow{\theta_2} 2\mathcal{X}_2,\\
     \mathcal{X}_2 &\xrightarrow{\theta_3} \emptyset.
 \end{aligned}
\end{equation}
The parameters ${\boldsymbol{\theta}} = \{ \theta_1, \theta_2, \theta_3\}$ control the events described above, and have corresponding rates $\theta_1 \mathcal{X}_1, \theta_2 \mathcal{X}_1 \mathcal{X}_2, \theta_3 \mathcal{X}_2$ respectively. 
\subsubsection{Experimental Setup}
% K = 5
% G = 8
% N = 1000
% \delta = 0.05
We study the inference using a total of $G=8$ rounds of refining the posterior estimate, with five repetitions using different random number seeds, setting the total amount of parameter proposals in each round to  $N=1000$. Further, we define the number of bins to $B=5$ per dimension and the threshold to be $\delta = 0.05$.
The LV model data consists of 51 observations in the time span $[0 \text{ h},50 \text{ h}]$. We construct the model and the corresponding simulator using \textit{GillesPy2}\footnote{https://github.com/StochSS/GillesPy2} \cite{gillespy_2017}. 
Finally, the initial conditions include $\mathcal{X}_1=50$ and $\mathcal{X}_2=100$, and the actual parameters to be inferred are $[log(1.0), log(0.005), log(1.0)]$. The prior we apply per dimension is $\mathbb{U}(log(0.002), log(2))$. 
For the ABC-SMC method, we use no summary statistics, i.e., the data itself is directly compared, and a Euclidean distance as the discrepancy measure.

\subsubsection{Results}
% MSE
Compared to the MA(2) model, we do not have an analytical posterior to compare to, and we cannot refer to the EMD of the method. However, we solve the problem with known truth (the parameters to infer) and consider the mean square error (MSE) for computed posteriors. The reasoning is that MSE holds information both about the bias and the variance of the distribution. In Figure~\ref{fig:MSE_LV}, we evaluate the said MSE of the BCNN approach together with SNPE-C and ABC-SMC. The BCNN performs on par with SNPE-C while seemingly holding to the posterior it finds; see round 8 and the spike in the variance of SNPE-C. 
Exploring what might have caused the increase in variance, we refer to Figure~\ref{fig:grid_LV}, which presents snapshots of a sequence of rounds from one run with the different methods. SNPE-C encounters an instability after round 4. From the MSE, we note that SNPE-C is mostly, not always, stable compared to the empirically robust BCNN.
Finally, we recognize that the initially poor performance of the ABC-SMC method can be attributed to the lack of adequately tuned summary statistics.

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_mse}}
\caption{The lines with error bars show the mean MSE with variability $\pm1$ std, recovered from five seeds with eight rounds for SNPE-C, BCNN, and ABC-SMC. All methods converge to similar MSE at the final round, with low variability. Notable is the initial increase in MSE for ABC-SMC; from Figure~\ref{fig:grid_LV} it can be inferred that the approximated posterior is bi-modal before finding a suitable discrepancy threshold.}
\label{fig:MSE_LV}
\end{center}
\vskip 0.2in
\end{figure}

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_dens}}
\caption{Snapshots of the posterior estimates for the LV problem using BCNN, SNPE-C, and ABC-SMC. Each column is a different round: 2, 4, and the final 8, and each row gives the three parameters combinations possible of $\theta_1$, $\theta_2$, and $\theta_3$. The true parameter value is indicated by the orange cross. The prior covers the entire box uniformly.}
\label{fig:grid_LV}
\end{center}
\vskip 0.2in
\end{figure}


%\subsection{Example 3?}
% placeholder
\section{Discussion}
\label{sec:discussion}
% Gotchas and pitfalls
%\textcolor{red}{PS: Can we mention some pitfalls, something to be careful about and pointers on how to get the best out of our approach?}
% - by using classification we cannot re-use the trained network, we have to start anew each round. Meaning that the information gained from each trained network must be leveraged to it's fullest when constructing the proposal distribution.
% - number of parameter dimensions

The results presented in the previous section demonstrate that the proposed approach is comparable to SNPE-C, which is a state-of-the-art method for approximating flexible posterior densities using NNs. The uncertainty estimates and a simplification strategy of accomplishing regression via classification using the BNN allows the adaptive sampling algorithm to consistently drive the search for the true posterior in the right direction. This can be seen in the very low standard deviations presented in Figures~\ref{fig:emd_MA2} and \ref{fig:MSE_LV}. 
However, an important observation is that SNPE-C does not utilize any exploitation similar to $\delta$ in the proposed approach  (see Section~\ref{sec:method}). Effectively, the deviations in performance are reduced using this exploitation. 

However, incorporating a similar exploitation strategy used in SNPE methods may not alone resolve the issue of occasional, substantially inaccurate posterior estimates, such as SNPE-C(2) in Figure \ref{fig:grid_MA2}. The stability and robustness of the proposed approach stems from the Bayesian setting considered in this work. Further, the BNN is specifically tailored for models with complex data distributions, such as discrete stochastic and dynamical models. The BNN functions as both  an embedding network to extract summary statistics and as a posterior estimator. However, the use of categorical distributions as posterior estimators is limited to fairly simplistic posteriors. For flexible posterior estimators we recommend SNPE-C, but to utilize the robustness and the exploitative properties of the proposed approach one might consider using both approaches in conjunction with each other. For example, using the BNN as a pre-processing step to robustly define a prior distribution, and subsequently passing the posterior estimate $p(\boldsymbol{\theta}'|y_0)$ as the prior for SNPE-C. This approach is outlined in Algorithm~\ref{alg:proposed}.

\begin{algorithm}[]
   \caption{Proposed: BNN in conjuction with SNPE-C}
   \label{alg:proposed}
\begin{algorithmic}
   \STATE Do Algorithm~\ref{alg:posterior} to get $\tilde{p}_G(\boldsymbol{\theta})$
   \STATE Do SNPE-C with $\tilde{p}_G(\boldsymbol{\theta})$ as prior
\end{algorithmic}
\end{algorithm}




% Significance of uncertainty estimates provided by the B(C)NN
%The results presented in the previous section underscore the importance of accounting for embedding model uncertainty towards the robustness of resulting LFI quality. The uncertainty estimates provided by the BNN allow the adaptive sampling algorithm to consistently drive the search for the true posterior in the right direction. This can be seen in the very low standard deviation of parameter inference error shown in Figures~\ref{fig:emd_MA2} and \ref{fig:MSE_LV}.

% Significance of adaptive sampling and categorical classification
%Further, adaptive sampling also enables consistent and stable refinement of the estimated posterior over successive rounds of algorithm execution. \textcolor{red}{PS: @Fredrik: pls put in some text here connecting how adaptive sampling and categorical classification is responsible for the refinement. Further, would be good to contrast with SNPE and talk about our advantages}

% Dynamical models
%\textcolor{red}{PS: why is our approach well-suited for dynamical models}
% + find "optimal" summaries, which is a hard task for time-series/dynamical models. Able to handle high dimension data well. 
\subsection{Future work}
% Future work
%\textcolor{red}{PS: Future work, explore more sophisticated sampling algorithms and the interplay between exploration and exploitation}

The convergence plots in Figure~\ref{fig:emd_MA2} and \ref{fig:MSE_LV} point towards an opportunity  for early stopping when the posterior estimate no longer receives significant updates. A possibility could be to measure the highest density region (HDR)~\cite{HDR}, or EMD, change between posterior estimates, and stop if the difference is below a threshold value. Using this could substantially reduce the number of simulations involved and remove the requirement of specifying the amount of rounds $G$.

Optimizing the number of bins $B$ as well as the number of proposals $N$ in each bin (support) is currently not trivial. A sufficient support for dynamical models is highly dependent on the response of simulation outputs to perturbations in the parameter space. For the MA(2) model, a larger $N$ per round was required, as well as fewer bins compared to the settings for the LV problem. This was owing to the MA(2) model mapping being ill-discriminated in nature as described earlier. Thus, for the BNN classifier to sufficiently learn discriminating features, one would require more data. Further, currently the bin width is fixed and does not consider the distribution of proposals. Exploring variable bin widths based on the rate of change of proposal distributions over successive rounds could potentially improve convergence speed, and also balance the support for bins more evenly. 

The procedure of assigning a continuous $\theta_i$ into a bin does not take discretization uncertainty into account (e.g., when $\theta$ is close to the egde of two bins), which can lead to inaccurate training. The aforementioned problems will be considered as future work and will involve adaptive bin selection based on the discriminative ability of the BNN. Further, bin uncertainties could potentially be tackled using fuzzy set theory and other bin strategies used in regression via classification.

The hyperparameter $\delta$ (see Section~\ref{sec:method}) that controls exploitation is currently considered to be fixed. However, an adaptive $\delta$ dependent on the current round can potentially improve convergence, e.g., an exponentially decaying $\delta$. The similarity to the threshold one use in ABC-SMC, aligns the adaptivity of $\delta$ well with strategies already in practice.

 %\textcolor{red}{FW: please elaborte, I don't understand}
 %One could potentially consider making it an adaptive procedure, dependent on how well the BNN resolves the previous step; if one observes a low effective sample size, then decrease the number of bins and vice versa. Additionally, currently, the bins are of equal width per dimension; by leveraging information from the proposal distribution, one could explore more informative bin widths, potentially bridging learning between rounds.


%discussion about early stopping, constructing bins, number bins and suggested exploitation. (the number of weights in BNN increase as number of bins increase, fuzzy bins?



\section{Conclusion}
\label{sec:conclusion}
This paper presented a novel likelihood-free parameter inference framework for parameter inference of dynamical simulation models. The proposed approach leverages Bayesian neural networks for learning high-quality summary statistics as the estimated posterior while also incorporating epistemic uncertainties. This allows for a posterior distribution estimate without the need for a post-processing step such as rejection sampling. An adaptive sampling algorithm utilizes the estimated posterior and uncertainty estimates from the Bayesian neural network to iteratively shrink the prior search space for the actual posterior distribution. An efficient classification-based density estimation approach further approximates the posterior. The proposed approach is compared to the current state-of-the-art sequential neural posterior estimation method and the well-established approximation Bayesian computation sequential Monte Carlo likelihood-free inference methods. Experiments performed on benchmark test problems show that the proposed approach is consistently more robust under various situations compared to existing methods while delivering comparable parameter inference quality.

\bibliography{references}
\bibliographystyle{icml2021}

\end{document}

%Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space. Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account.This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions.An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network.This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.


% This document was modified from the file originally made available by
% Pat Langley and Andrea Danyluk for ICML-2K. This version was created
% by Iain Murray in 2018, and modified by Alexandre Bouchard in
% 2019 and 2021. Previous contributors include Dan Roy, Lise Getoor and Tobias
% Scheffer, which was slightly modified from the 2010 version by
% Thorsten Joachims & Johannes Fuernkranz, slightly modified from the
% 2009 version by Kiri Wagstaff and Sam Roweis's 2008 version, which is
% slightly modified from Prasad Tadepalli's 2007 version which is a
% lightly changed version of the previous year's version by Andrew
% Moore, which was in turn edited from those of Kristian Kersting and
% Codrina Lauth. Alex Smola contributed to the algorithmic style files.

suppl.tex
\documentclass{article}

% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{rotating}
\usepackage{booktabs} % for professional tables

% Added by authors
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{pdflscape}

% hyperref makes hyperlinks in the resulting PDF.
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
% please comment out the following usepackage line and replace
% \usepackage{icml2021} with \usepackage[nohyperref]{icml2021} above.
\usepackage{hyperref}

% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Use the following line for the initial blind version submitted for review:
\usepackage{icml2021}

% If accepted, instead use the following line for the camera-ready submission:
%\usepackage[accepted]{icml2021}

% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{Robust and integrative Bayesian neural networks for likelihood-free parameter inference
}

\begin{document}

\onecolumn
%\icmltitle{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\icmltitle{Robust and integrative Bayesian neural networks\\ for likelihood-free parameter inference\\
Supplementary Material}

\section{Extended Experiments for the MA(2) Test Problem:}
We investigate the effect of varying random seeds, and a comparison of execution times for the proposed approach against SNPE-C. We also explore the effect of varying the hyperparameters involved in the proposed BCNN-based approach.
\subsection{Qualitative evaluation with additional seeds}

\begin{figure*}[ht]
\begin{center}
\centerline{\includegraphics[width=\textwidth]{img/ma2_suppl_grid.pdf}}
\caption{Ten individual seeds for estimating posterior using SNPE-C (A) and BCNN (B). }
\label{fig:grid_LV}
\end{center}
\end{figure*}


\twocolumn
In order to investigate the robustness of the proposed approach, ten distinct random seeds were used in the parameter inference task for the BNN (BCNN) and SNPE-C based methods. Figure~\ref{fig:grid_LV} depicts the results with each row representing a seed, and each column representing a round in the sequential parameter inference process. The random walk Metropolis-Hastings MCMC method was used to calculate the exact posterior for the purpose of comparisons with BCNN and SPNE-C. 

It can be observed that the proposed BCNN-based approach is consistent across different seeds, while SNPE-C shows susceptibility in three cases (seeds 4,5 and 9). It can also be seen that SNPE-C typically starts with a tighter estimated posterior in the first round, but relative gains over subsequent rounds are small. In comparison, BCNN is slower in the initial stages, with larger estimated posteriors in the first round. However, relative gains over successive rounds are more significant with substantial refinement being achieved. This can be attributed to the exploitation offered by the binning-based classification approach involved in parameter inference using BCNN. The final estimated posterior in the sixth round is more concentrated for the BCNN as compared to SNPE-C. 

\subsection{Embedding Network architectures}
\label{sec:arch_ma2}
The BCNN was implemented using the TensorFlow library (Keras and tensorflow probability), while the SBI package (SNPE-C) makes use of PyTorch. The architectures are shown in Table \ref{tab:ma2}.
\begin{table}[ht]
\caption{Architecture for the BCNN (top) and SNPE-C (bottom); NDE refers to neural density estimator.}
\label{tab:ma2}
%\vskip 0.15in
\begin{center}
%\begin{small}
%\begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 100, 1)]&0\\
Conv1DFlipout (ReLU) & (None, 100, 6)  &66\\
MaxPooling1D    & (None, 10, 6)&0\\
Flatten    & (None, 60)&0\\
DenseFlipout (Softmax)     &(None, 16)&1936\\
\midrule
Total weights &&2002

\end{tabular}
%\end{sc}
%\end{small}
%\end{center}

%\caption{Architecture SNPE-C, icluding embedded summary statistics network and neural density estimator (NDE) with 10 hidden units and 2 transforms}
% \label{sample-table}
% \vskip 0.15in
% \begin{center}
% \begin{small}
% \begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 100, 1)]&0\\
Conv1d (ReLU) &(None, 100, 6)  &36\\
MaxPooling1d    &(None, 10, 6)&0\\
view (Flatten)    & (None, 60)&0\\
Linear (ReLU)     &(None, 16)&488\\
NDE (MAF) &10 hidden, 2 transform&970\\
\midrule
Total weights &&1494

\end{tabular}
%\end{sc}
%\end{small}
\end{center}
%\vskip -0.1in
\end{table}

\subsection{Hyperparameters of the BNN}
Here we experimented with different hyperparameters for the BNN (BCNN), including different number of bins (classes) in Figure~\ref{fig:ma2_bins}, and different exploitation strategies using the threshold $\delta$ in Figure~\ref{fig:ma2_thresh}. For MA(2), four bins per parameter (16 classes) is the best performing setting, however using five bins (25 classes) results in similar performance. For the exploitation threshold $\delta$, we observe the best overall performance using $\delta=0.05$ when observing log mean EMD. For mean MMD both show similar results. Additionally, a proposed strategy of using an exponentially decaying threshold, i.e., reducing the amount of exploitation per round, performs on a level in between $\delta = 0.05$ and $\delta = 0.0$. 
\begin{figure}[ht]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_bins__4.25x2_fontscale08.pdf}}
\caption{Log mean EMD and mean MMD for BCNN with different number of classes corresponding to the number of bins.}
\label{fig:ma2_bins}
\end{center}
\end{figure}

\begin{figure}[ht]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_thresh__4.25x2_fontscale08.pdf}}
\caption{Log mean EMD and mean MMD for BCNN for different threshold $\delta$. $\delta = 0.0$ refer to no threshold being used (i.e no exploitation). $\delta = 0.05$ is the strategy used in the main article. $\delta = e$ is an exponentially decaying threshold.}
\label{fig:ma2_thresh}
\end{center}
\end{figure}

\subsection{Execution time}
Figure~\ref{fig:ma2_time} shows the time difference for the inference task between the two implementations used in this study. Note that the BCNN used for MA(2) is one joint model (2D bins), however for the LV model several co-variate BCNN models are trained.

It is important to note that differences in software implementations (TensorFlow versus PyTorch) and approaches to parallelization make direct empirical comparisons challenging. Further, factors such as the early stopping criterion will also have an effect on the result.

%Note that this is by no means a final evaluation of the computational complexity between BCNN and SNPE-C. Both have different implementations and are using various parallel and neural network packages, further factors such as early stopping criterion will also have an effect on the result. However, Figure~\ref{fig:ma2_time} shows the time difference for the inference task between the two implementations used in this study. Observe that the BCNN used for MA(2) is one joint model (2D bins), however for the LV model several co-variate BCNN models are trained. 

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_time__4.25x2_fontscale08.pdf}}
\caption{(left) Time (minutes) per round between BCNN and SNPE-C using MAF. (right) Cumulative time (minutes). Both include simulation time per round (3000 simulations/round).  }
\label{fig:ma2_time}
\end{center}
\end{figure}

%\subsection{Various neural density estimators in SNPE-C}
%\begin{figure}[ht]
%\vskip 0.2in
%\begin{center}
%\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_NDE__4.25x2_fontscale08.pdf}}
%\caption{Text}
%\label{fig:ma2_time}
%\end{center}
%\vskip 0.2in
%\end{figure}




\section{Extended Experiments for the Lotka-Volterra (LV) Test Problem:}
The structure of extended experiments for the LV test problem follows the MA(2) setup in the text above. 
%We investigate the effect of varying random seeds, a comparison of execution times for the proposed approach against SNPE-C. We also explore the effect of varying the hyperparameters involved in the proposed BCNN-based approach.

\subsection{Qualitative evaluation with additional seeds}
%To investigate the robustness in the LFI-methods, we repeat the experiments multiple times, with five different seeds. 
The robustness of the three approaches - BCNN, SNPE-C and ABC-SMC against five different seeds is now investigated for the Lotka-Volterra predator-prey model.
In Figure~\ref{fig:lv_grid}, we show the snapshots and the final two-dimensional marginal distributions for the five independent runs. Comparison of the three approaches shows that the BCNN approach is relatively robust, while the ABC-SMC method converges slowly. It can also be seen that SNPE-C often converges to a different posterior shape as compared to BCNN and ABC-SMC. 



% \begin{figure}[ht]
% \vskip 0.2in
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_0.pdf}}
% \caption{Snapshot \# 1 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_0}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_1.pdf}}
% \caption{Snapshot \# 2 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_1}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_2.pdf}}
% \caption{Snapshot \# 3 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_2}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_3.pdf}}
% \caption{Snapshot \# 4 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_3}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_4.pdf}}
% \caption{Snapshot \# 5 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_4}
% \end{center}
% \vskip 0.2in
% \end{figure}

%\twocolumn
\subsection{Embedding Network architectures}
%The BCNN was implemented in TensorFlow (Keras and tensorflow probability) while the SBI package (SNPE-C) makes use of PyTorch. 
Implementations follow the description in Section \ref{sec:arch_ma2}. The architectures are shown in Table \ref{tab:lv}.
\begin{table}[ht]
\caption{Architecture BCNN (top) and SNPE-C(bottom).}
\label{tab:lv}
%\vskip 0.15in
\begin{center}
%\begin{small}
%\begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 51, 2)]&0\\
Conv1DFlipout (ReLU) &(None, 51, 20)  &260\\
MaxPooling1D    &(None, 10, 20)&0\\
Flatten    & (None, 200)&0\\
DenseFlipout (Softmax)     &(None, 25)&10025\\
\midrule
Total weights &&10285

\end{tabular}
%\end{sc}
%\end{small}
%\end{center}

%\caption{Architecture SNPE-C, icluding embedded summary statistics network and neural density estimator (NDE) with 10 hidden units and 2 transforms}
% \label{sample-table}
% \vskip 0.15in
% \begin{center}
% \begin{small}
% \begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 51, 2)]&0\\
Conv1d (ReLU) &(None, 51, 20)  &140\\
MaxPooling1d    &(None, 10, 20)&0\\
view (Flatten)    & (None, 200)&0\\
Linear (ReLU)     &(None, 25)&1608\\
NDE (MAF) &10 hidden, 2 transform&912\\
\midrule
Total weights &&2660

\end{tabular}
%\end{sc}
%\end{small}
\end{center}
%\vskip -0.1in
\end{table}

\subsection{Hyperparameters of BNN}
As in the case of the MA(2) test problem, the hyperparameters of interest to explore in the context of the BNN aproach for LV include the number of bins (classes) and the extent of exploitation controlled by the threshold $\delta$. In Figure~\ref{fig:lv_bins}, we explore three values for the bins. As the number of bins for the discretization of continuous data increases, so does the convergence speed towards a lower MSE.
%One could theorize that they all will converge to the same MSE, however, at different rates. 

In Figure~\ref{fig:lv_thresh}, we compare the MSE for 8 rounds using a threshold value of $\delta = 0.05$ and $\delta = 0$, i.e., with or without exploitation. It can be seen that incorporating exploitation via $\delta > 0$, results in a more rapid decline in MSE as compared to the case of no exploitation. This is expected, validates the purpose of exploitation in the proposed approach. 
%As in the case of the number of bins, the two cases might converge to the same MSE; however, when using exploitation $\delta > 0$, predictably, one reaches a lower MSE faster than without.

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_bins.pdf}}
\caption{MSE for BCNN with different number of classes corresponding to the square of the number of bins per parameter, on the LV test problem.}
\label{fig:lv_bins}
\end{center}
\end{figure}

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_thresh.pdf}}
\caption{MSE for BCNN when using threshold $\delta$: $0.05$ and $0.0$, on the LV test problem.}
\label{fig:lv_thresh}
\end{center}
\end{figure}

\subsection{Execution time}
Figure~\ref{fig:lv_time} presents the execution time of the BCNN and SNPE-C approaches. However, we note that several factors affect this comparison. It can be observed that the BCNN approach is slower as compared to SNPE-C owing to the overhead of training times for multiple classifiers.
%One should observe the execution time presented in Figure~\ref{fig:lv_time}  with caution. 
The number of required classifiers depends upon the number of parameter pair-combinations, e.g., three parameters requires four models, four parameters require six models, etc. The additional training time for the BCNN in case of LV can be reduced via better parallelization of the training process of the three classifiers. Please refer to Figure~\ref{fig:ma2_time} for a comparison of execution times for a single trained model.

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_time.pdf}}
\caption{Time per round (minutes) and cumulative time (minutes), for BCNN and SNPE-C, on the LV method.}
\label{fig:lv_time}
\end{center}
\end{figure}


\newpage
\onecolumn
%\begin{landscape}
\begin{sidewaysfigure}[ht]
\centering
\includegraphics[width=21cm]{img/LV_grid_seeds.pdf}
\caption{Snapshot \# 1-5 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
\label{fig:lv_grid}
\end{sidewaysfigure}
%\end{landscape}
\end{document}
toArxiv.tex
%%%%%%%% ICML 2021 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%%

\documentclass{article}

% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{booktabs} % for professional tables

% Added by authors
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{mathtools}


\usepackage{rotating}
\usepackage{pdflscape}



% hyperref makes hyperlinks in the resulting PDF.
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
% please comment out the following usepackage line and replace
% \usepackage{icml2021} with \usepackage[nohyperref]{icml2021} above.
\usepackage{hyperref}

% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Use the following line for the initial blind version submitted for review:
%\usepackage{icml2021}

% If accepted, instead use the following line for the camera-ready submission:
\usepackage[accepted]{icml2021_arxiv}

% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{Robust and integrative Bayesian neural networks for likelihood-free parameter inference}

\begin{document}

\twocolumn[
%\icmltitle{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\icmltitle{Robust and integrative Bayesian neural networks\\ for likelihood-free parameter inference}
% It is OKAY to include author information, even for blind
% submissions: the style file will automatically remove it for you
% unless you've provided the [accepted] option to the icml2021
% package.

% List of affiliations: The first argument should be a (short)
% identifier you will use later to specify author affiliations
% Academic affiliations should list Department, University, City, Region, Country
% Industry affiliations should list Company, City, Region, Country

% You can specify symbols, otherwise they are numbered in order.
% Ideally, you should not use this facility. Affiliations will be numbered
% in order of appearance and this is the preferred way.
\icmlsetsymbol{equal}{*}

\begin{icmlauthorlist}
\icmlauthor{Fredrik Wrede}{equal,uu}
\icmlauthor{Robin Eriksson}{equal,uu}
\icmlauthor{Richard Jiang}{ucsb}
\icmlauthor{Linda Petzold}{ucsb}
\icmlauthor{Stefan Engblom}{uu}
\icmlauthor{Andreas Hellander}{uu}
\icmlauthor{Prashant Singh}{umu}
\end{icmlauthorlist}

\icmlaffiliation{uu}{Department of Information Technology, Uppsala University, Uppsala, Sweden}
\icmlaffiliation{ucsb}{Department of Computer Science, University of California-Santa Barbara, Santa Barbara, CA, USA}
\icmlaffiliation{umu}{Department of Computing Science, Umeå University, Umeå, Sweden}

\icmlcorrespondingauthor{Fredrik Wrede}{fredrik.wrede@it.uu.se}
\icmlcorrespondingauthor{Prashant Singh}{prs@cs.umu.se}

% You may provide any keywords that you
% find helpful for describing your paper; these are used to populate
% the "keywords" metadata in the PDF but will not be shown in the document



\icmlkeywords{Bayesian Approximate Inference, Active Learning, Supervised Learning, Computational Biology, Bayesian Deep Learning}

\vskip 0.3in
]

% this must go after the closing bracket ] following \twocolumn[ ...

% This command actually creates the footnote in the first column
% listing the affiliations and the copyright notice.
% The command takes one argument, which is text to display at the start of the footnote.
% The \icmlEqualContribution command is standard text for equal contribution.
% Remove it (just {}) if you do not need this facility.

%\printAffiliationsAndNotice{}  % leave blank if no need to mention equal contribution
\printAffiliationsAndNotice{\icmlEqualContribution} % otherwise use the standard text.
\begin{abstract}
State-of-the-art neural network-based methods for learning summary statistics have delivered promising results for simulation-based likelihood-free parameter inference.
Existing approaches for learning summarizing networks are based on deterministic neural networks, and do not take network prediction uncertainty into account. 
This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and produces a proposal posterior density using categorical distributions. 
An adaptive sampling scheme selects simulation locations to efficiently and iteratively refine the predictive proposal posterior of the network conditioned on observations.
This allows for more efficient and robust convergence on comparatively large prior spaces. 
The approximated proposal posterior can then either be processed through a correction mechanism to arrive at the true posterior, or be used in conjunction with a density estimator to obtain the posterior.
We demonstrate our approach on benchmark examples.% and compare against related methods.
% Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space.
% % ** 1 **
% Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account. 
% This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions.
% %This work proposes an uncertainty-enhanced summary statistics learning framework using Bayesian neural networks, which incorporates epistemic uncertainties to quantify the confidence of the trained model. 
% % ** 2 **
% An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network.
% %conditioned on observations, hence eliminating the need for discrepancy-based rejection or density estimation and reducing the number of simulations required. 
% % ** 3 **
% %Our novel approach also includes estimating the posterior using categorical distributions, subsequently transforming the learning task into a classification problem. 
% This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.
\end{abstract}

\section{Introduction}
\label{sec:intro}
 Likelihood-free inference (LFI) is a class of methods designed for parameter inference of models with intractable or expensive likelihoods. 
 %The type of models range from ... 
 Approximations to the posterior distribution conditioned on observed data are made with the aid of forward simulations of the model, thus also referred to as simulation-based inference~\cite{Cranmer_frontierSBI}. Approximate Bayesian computation (ABC)~\cite{sisson2018handbook} is a popular method in this setting, and involves the use of a discrepancy measure to compare simulated data against observed data. A threshold on the discrepancy measure decides whether a proposed parameter can be accepted as a sample from the posterior. 
 
While this approach has shown to be robust, it has certain drawbacks. For many dynamical models, the simulation output is high-dimensional, 
and one uses summary statistics as a dimension reduction technique
 % avoid large approximation errors
 \cite{sisson2018handbook,wood2010statistical,engblom2020bayesian}. Selecting informative summary statistics so that the discrepancy measure can accurately accept posterior samples is often a tedious task, typically requiring detailed domain knowledge together with a trial and error approach. 
 
 An alternative approach to handpicking summary statistics is to utilize embedded neural networks (NN) to learn summary statistics from simulated data \cite{jiang2017learning, dinev2018dynamic, wiqvist2019partially, aakesson2020convolutional,chen2020neural}.  Some of these methods require a post-processing ABC scheme to yield the posterior, others work in conjunction with neural density estimators \cite{lueckmann2021benchmarking} to either estimate the posterior directly, or use synthesized likelihoods which requires Markov chain Monte Carlo (MCMC) sampling. Although deterministic NNs for learning summary statistics have shown impressive results, they are prone to be inaccurate in regions where the training data is sparse or not well represented (particularly in the case of problems with high-data dimensionality) \cite{aakesson2020convolutional}. 
 %The uncertainty of prediction or network prediction confidence is not taken into account in the LFI pipelines.
 %they can often lead to too `overconfident' predictions\textcolor{red}{cite}. Although implementations of embedding networks include regularization techniques to avoid overfitting, they still suffer from inaccuracies in regions of the parameter space not well-represented in the training data. 
Deterministic NNs do not directly provide an estimate of their prediction uncertainty, which can potentially mislead the parameter inference process.
%Deterministic NNs cannot quantify uncertainties associated with unseen data independently, which manifests as an inherent problem when one samples a finite amount of parameter proposals from large, potentially uninformative priors. 
This lack of direct uncertainty quantification motivates the design of more robust and expressive embedding networks, that can drive the LFI process in a more reliable manner.
 
 % ToDo for PS: elaborate more on the overconfident part - this is due to (limited) sampling in hd spaces, with large gaps of no data
 % PS: Perhaps we should clearly say - Approach 1: (BNN) and Approach 2: (BNN + classification)
%This paper presents a novel likelihood-free parameter inference framework that improves upon two characteristics of embedded NN-based LFI methods, as outlined below.

Bayesian embedded NNs are proposed as summary statistics that provide prediction uncertainties, and coupled with a distribution estimator, also approximate the proposal posterior. An adaptive sampling algorithm utilizes these uncertainty estimates to iteratively refine the approximated proposal posterior. Including prediction uncertainty in the LFI process provides robustness against potential gaps in training data.

The true posterior distribution can be obtained from a correction procedure applied to the proposal posterior (out of scope of this work). The need for the correction arises due to the iterative nature of the adaptive sampling algorithm. The proposal posterior is refined in each iteration by sampling solely within the proposal posterior of the previous iteration. Therefore, in order to account for \emph{proposal leakage} or regions within the true posterior missed due to inaccuracies in the NN, a correction mechanism is needed that enlarges the proposal posterior accordingly.

%This work explores Bayesian embedded networks as trainable summary statistics that provide prediction uncertainties, and directly estimate the posterior distribution.
%Introducing epistemic uncertainties in the embedded network will enable a more robust LFI workflow by considering model confidence in addition to model predictions. The embedded network's predictive posterior will directly estimate the posterior distribution for the inference task, ultimately integrating embedded summary statistics networks with neural density estimators. 
The proposed approach involves approximating the proposal posterior through discrete categorical distributions, subsequently solving a regression problem via classification. This approach in similar form has delivered encouraging results~\cite{weiss1995rule, torgo1996regression, AHMAD_RvC}. The main motivation behind this design is that performing regression with high-dimensional inputs (such as dynamical simulation responses) to continuous multi-dimensional outputs (such as parameters) is a highly challenging task. Even though NNs have the learning capacity to do so, generalizing the networks often proves difficult. 
%Particularly if the training data is not fixed over individual training procedures, such as the case for parameter inference. It's our intention to make parameter inference using NNs more robust by utilizing regression via classification.     

 We demonstrate the efficiency and robustness of our approach on benchmark problems, and also compare the proposal posteriors against approximated posteriors obtained using deterministic NNs paired with neural density estimators. Experimental results show  that the proposed approach is consistently more robust than existing methods, with relatively small variance in parameter inference quality performance. The error in parameter estimation even without the correction process is comparable to existing methods. %\textcolor{red}{Think more about how we add value over SNPE}
 %As an alternative approach, some methods have been focusing on introducing inverse mapping functions $f$.  By fitting the inverse mapping function $f(\bold{y}, w) \mapsto \hat{\theta}$ using simulated data  $X = \{y_i, \theta_i\}^N$, a loss function $L(\theta_i, \hat{\theta_i})$ can be optimized w.r.t $w$ between predicted and true parameters, the weights $w$ should ultimately incorporate the discrepancy between samples. While some variants can directly yield density estimators $f(y_0,w) \mapsto \hat{p}(\theta|y_0}$, others use it in a post-processing rejection scheme by letting $d(f(y_i,w), f(y_0,w))$. Learning the inverse mapping function is typically referred as a regression problem where one of the most popular approaches is to use artificial neural networks (ANN).}

%However, most applied deep learning architectures are deterministic and non-probabilistic and can yield overconfident predictions on targets. Further, classification as shown to be superior to regression in prediction performance when $f$ inherits complex non-linear patterns. Thereof, we explore using Bayesian neural networks to better account for epistemic uncertainty and regularization in the mapping function and we discretize $\theta$ space into categorical classes to transform the problem into a classification task, assuming that sample paths $y$ to be similar when in parameter proximity. The inherent properties of dynamical processes makes convolutional neural networks (CNN) an attractive option for learning the mapping function $f$. CNNs make use of a filtering kernel to create local feature maps.Previous studies have demonstrated the capabilities of using determinstic CNNs for likelihood-free parameter inference \cite{}. In this study we use a similar architecture but make use of probablistic layers in Tensflow probability package to place parameterized distributions over the weights of the neural network and apply variational inference using evidence lower bound (ELBO) and \textit{Bayes by Backprop} to fit a posterior distribution. This approach suggests that, not one, but an ensemble of neural networks are learned, where each network's weights are sampled from the fitted posterior. While it might seem computationally heavy learning an infinite ensemble of networks, in fact the amount of weights only increases by a factor of two when using Monto Carlo estimates of the gradients on an otherwise intractable problem.
We organize the paper as follows: Section~\ref{sec:background} introduces the problem setting, the notation, and related work. Section~\ref{sec:method} presents the proposed LFI parameter estimation approach. Section~\ref{sec:experiments} demonstrates the proposed methodology on benchmark examples and draws comparisons to other methods. Section~\ref{sec:discussion} presents a discussion on the merits and applicability of the approach. Finally, Section~\ref{sec:conclusion} gives concluding remarks about the novel method presented in the paper.

\section{Background} %  notation, estimate PDF with Neural net (with embedding net)
\label{sec:background}
% problem set-up | we want the posterior given model M, and observation y0
Let ${\bf y}_0$ be a target observation which can be approximately modeled by $M({\bf y} | {\boldsymbol \theta})$.
%Assume a dynamical model  $\mathcal{M}$ which has generated an observation ${\bf y}_0$. Next, assume some model error by approximating the model by $M({\bf y} | {\boldsymbol \theta})$, with parameters $\boldsymbol \theta$.
Forward simulation of the model $M$ results in the sample path ${\bf y}^{1:T}$ henceforth referred to as ${\bf y}$.
%Let $ {\bf y}_0 \sim M({\bf y}| {\boldsymbol \theta})$ be observed data approximately sampled from $M({\bf y}|\boldsymbol{\theta})$, i.e., we assume some model error.
%The observed data may also have been obtained from the physical process corresponding to $M$. 
The objective is to infer the parameters conditioned on the observation, i.e., the posterior density $p(\boldsymbol{\theta} | {\bf y}_0) \propto p({\bf y}_0 | \boldsymbol{\theta})p(\boldsymbol{\theta})$. However, as we assume that the likelihood is computationally intractable, which is often the case for dynamical models, an estimate $\hat{p}(\boldsymbol{\theta} | {\bf y}_0)$ of the true posterior distribution must be approximated. Approaches in this setting typically utilize access to the observed data  ${\bf y}_0$ and the ability to perform simulations from model $M$ at well-chosen parameter locations. We summarize some the methodologies under the supposed setting below.
%, and motivate the need for a fully Bayesian model-driven LFI approach.


 
 

\subsection{Related work}
% --- Simulation based Inference --- %
% discrepancy-, SS based
The ABC family of methods forms the most popular and established LFI approach (introduced in Section~\ref{sec:intro}). 
%The methodology consists of a sample-simulate-compare cycle where parameter samples $\boldsymbol{\theta}_i \sim p(\boldsymbol{\theta})$ drawn from a defined prior distribution $p(\boldsymbol{\theta})$ are simulated using the model $M$ to obtain sample paths ${\bf y}$, which one consequently compare to observed data ${\bf y}_0$ to either declare T1 as being accepted as part of the estimated posterior or rejected. The workflow consists of several hyperparameters, including the choice of the prior P1, dimension reducing summary statistics (when ${\bf y}$ is high dimensional, e.g., time series), a distance or discrepancy measure to compare simulated and observed data, and a tolerance bound $\epsilon$ to define acceptance/rejection. 
Although remarkably successful, ABC methods are typically challenging to setup optimally for accurate inference and convergence speed \cite{sisson2018handbook,marin2012approximate}. Extensions such as sequential Monte Carlo methods (ABC-SMC) \cite{sisson2007sequential} can potentially accelerate the convergence speed, but still require laborious hyperparameter selection. In particular, the choice of informative summary statistics, is still present, and is critical for practical and accurate parameter inference as it forms the means of comparing simulated proposals with the observation. 

Consequently, summary statistic selection has received considerable attention \cite{sisson2018handbook}[Chapter 5]. However, traditional summary statistics such as statistical moments may not always be expressive enough to capture intricate details in rich dynamical data. Further, ideal statistics may even be missing from candidate pool of statistics to select from, e.g., \emph{tsfresh}~\cite{christ2018time}. A recent development has been the use of data-driven regression models to extract features from data, and function as summary statistics. It has been shown that the ideal summary statistic is the posterior mean, when minimizing the quadratic loss \cite{fearnhead2012constructing}. In particular, NN based regression models have delivered highly promising results towards learning the estimated posterior mean as summary statistics from high-dimensional detailed time series \cite{jiang2017learning,wiqvist2019partially,aakesson2020convolutional}. Such models learn the mapping $F_{\bf w}({\bf y}) \mapsto \mathbb{E}[\boldsymbol{\theta}]$ which can be seen as an amortized point estimate for a given ${\bf y}$, where ${\bf w}$ are the weights of the regression model. However, an overhead associated with \emph{trainable} summary statistics is the need for large amounts of simulated training data to yield an accurate  posterior mean estimate. Although there are opportunities for re-using the training data in the later rejection sampling step to estimate the conditioned posterior, the sheer amount of training data required is problematic whenever the simulations are expensive to compute. % Mention that they can't quantify uncertainties about their estimates 

%Synthetic likelihood based approaches \cite{wood2010statistical}  step away from discrepancy distances and approximate the likelihood conditioned on the summary statistics.

% remove SS, still discrepancy-dependent
%Efforts to use embedding nets to compute optimal summarizing statistics have shown great promise, NN \cite{jiang2017learning}, Convolutional NN \cite{aakesson2020convolutional}, but they still necessitate a discrepancy distance to evaluate proposals. These approaches consists of fitting a regression model $F_w(y) \mapsto \theta$ parameterized by the weights $w$ and subsequently use prediction on simulated data
%\textcolor{red}{FW: need to introduce SNPE-C in particular\cite{greenberg2019automatic}}
Neural density estimators form a family of methods that directly estimate densities with the aid of NNs and have shown impressive performance on challenging problems \cite{lueckmann2021benchmarking}, such as the masked autoregressive flows (MAF) and the neural spline flow (NSF) \cite{papamakarios2017masked,NSF}. While some neural density estimators are used to estimate posterior densities \cite{APT_2019,SNPE_A_Papamakarios, SNPE_B_lueckmann2017flexible}, others rely on constructing synthesized likelihoods or ratios which require MCMC sampling to extract the posterior \cite{papamakarios2019sequential, lueckmann2019likelihood,SNRE_a, SNRE_b_constrative}. The formed posteriors are amortized, i.e., any observation can be conditioned on the posterior. 

However, a collection of algorithms based on neural density estimators uses sequential Bayesian updates to the posterior, conditioned on a particular target observation. These include sequential neural posterior estimation (SNPE) \cite{SNPE_A_Papamakarios, SNPE_B_lueckmann2017flexible, APT_2019},  sequential neural likelihood estimation (SNLE) \cite{papamakarios2019sequential, lueckmann2019likelihood} and sequential neural ratio estimation (SNRE) \cite{SNRE_a, SNRE_b_constrative}. A benchmarking effort was recently conducted comparing the performance of these approaches on various problems \cite{lueckmann2021benchmarking}. These approaches also support the use of embedding networks to turn high-dimensional simulation outputs to informative summary statistics. The weights of the embedding networks are then trained together with the weights of the neural density estimator. Thus, these shared architectures enable sequential updates on the conditioned posterior and the summary statistics learning process, ultimately reducing the amount of simulations required. 

SNPE-C (also known as automatic posterior transform - APT) \cite{APT_2019} is a recent SNPE technique that learns to infer the true posterior by maximizing an intermediate estimated proposal posterior. SNPE approaches form a good class of methods to compare the proposed approach to, as the use of sequential Bayesian updates to refine the estimated posterior is close in spirit to adaptive sampling used in this work. We note that SNPE approaches approximate the true posterior, while the proposed approach needs a correction step as post-processing to arrive at the true posterior from a proposal posterior. SNPE approaches have also been shown to perform well across a variety of test problems \cite{lueckmann2021benchmarking,APT_2019}. Among SNPE approaches, SNPE-C is superior as it is more flexible and scalable as compared to SNPE-A and SNPE-B \cite{APT_2019}, and is used for the purpose of comparisons in the experiments in this paper. 


%The most recent and superior version of SNPE is the Automatic Posterior Transform (APT) \cite{APT_2019}, referred as SNPE-C in the software package SBI\footnote{https://www.mackelab.org/sbi/} which is the naming we will be using henceforth.       
    
% A list, that then has to be made into a flowing text.
% *classification*
%Using classification in LFI is not novel~\cite{gutmann2018likelihood}.  
%
% *flow-based*
%Flow-based model $q(\eta|x)$ condition on $x$ to generate the next training data set $\mathcal{D}$ in order to approximate the posterior $p(\eta|x)$, APT (SNPE-C) \cite{greenberg2019automatic, gonccalves2020training}
%Flow-based model with $q(\eta|x)$ conditioned on $\eta$ \cite{papamakarios2019sequential}
%SNPE \cite{lueckmann2017flexible,lueckmann2019likelihood,papamakarios2017masked}
% --- collecting ---
%Benchmarking effort of SBI \cite{lueckmann2021benchmarking}, 

\subsection{Motivation for Bayesian neural networks}
We implement a NN for summary statistics, with density estimation through Bayesian neural networks (BNNs). The introduction of weight uncertainties in NNs has enabled a robust means for learning an ensemble of models \cite{Bayesbybackprop}, represented by the posterior density $p({\bf w}|{\bf y})$ where ${\bf w}$ are the network weights.
%Blundell \textit{et al.} demonstrate with their Bayes by backprop algorithm how one can obtain and fit an ensemble of models represented by the posterior density $p(w|X)$ where $w$ is the network weights and $X$ the training data. 
It has also been demonstrated \cite{Bayesbybackprop} that the number of weights only increases by a factor of two by utilizing Monto Carlo (MC) sampling of gradients when updating the weights. The benefit of using a BNN is that the proposal posterior predictive distribution $\tilde{p}_w(\boldsymbol{\theta}|{\bf y}) = \int \tilde{p}(\boldsymbol{\theta}|{\bf w}, {\bf y})p({\bf w}|{\bf y})d{\bf w}$ provides an uncertainty measure to its predictions. The uncertainty estimates form an elegant means of identifying and countering overfitting, which translates into more robust and efficient embedding networks. Different embedding network architectures such as RNNs or CNNs can be implemented in the BNN setting to provide prediction uncertainty estimates, and compute the proposal posterior $\tilde{p}(\boldsymbol{\theta}|{\bf y}_0) \coloneqq \tilde{p}_w(\boldsymbol{\theta}|{\bf y}_0)$ by the proposal posterior predictive distribution. We will henceforth denote $\tilde{p}_w(\boldsymbol{\theta}|{\bf y})$ simply as $\tilde{p}(\boldsymbol{\theta}|{\bf y})$. The uncertainty estimates also allow sequential adaptive sampling of the posterior, conditioned on a target observation for faster convergence, similar to SNPE. 

%However, due to the integral in the predictive posterior being analytically intractable, numerical integration needs to be utilized (e.g., using MC methods). For the application of LFI, a BNN will provide the flexibility of using embedding networks, e.g., architectures such as RNNs or CNNs, that provide uncertainty estimates of predictions, and directly estimate the posterior $p(\theta|y_0) \approx p(\theta|X)$ by the posterior predictive distribution. This, in turn will enable sequential adaptive sampling of the posterior conditioned on a target observation for faster convergence. 
%\textcolor{red}{FW: Not completely sure of the notation of posterior predictive distribution in our case is it $p(\theta|X)$, $p(y_0|X)$, $p(\theta|w,X)$ etc??} 
%\textcolor{blue}{PS: $p(\theta|X)$, or $p(\theta|w,X)$ (there are arguments on both sides whether or not to include $w$. I'd go with $p(\theta|X)$. It is okay to abstract model specifics (in the form of $w$).}
%The need for large quantities of (potentially computationally expensive) simulated data for training summary statistics and subsequently estimating the posterior distribution using the ABC family of methods can lead to impractical parameter inference processing times. Therefore, this paper aims at developing a data-efficient LFI parameter estimation pipeline that minimizes the training data needed to obtain accurate high-quality summary statistics using an adaptive sampling algorithm, and directly estimates the posterior using Bayesian neural networks thereby eliminating the need for ABC-based posterior estimation. Further, an efficient classification-based approach is also presented that iteratively shrinks the prior search space.
% 
% 

\section{Embedded Summary Statistics via Bayesian neural networks}
\label{sec:method}
 Let $D = \{({\bf y}_i,\boldsymbol{\theta}_i)\}_1^N$ consist of simulated data and parameter proposals sampled from the prior distribution over the parameter space $p(\boldsymbol{\theta})$. Consider a BNN, parameterized by the weights ${\bf w}$, that computes the posterior density $p({\bf w}|{\bf y})$ of a probabilistic model $\tilde{p}(\boldsymbol{\theta}|{\bf y},{\bf w})$ by placing a prior $p({\bf w})$ on the weights. The proposal posterior predictive distribution for new unseen sample path ${\bf y}_t$,  $\tilde{p}(\hat{\boldsymbol{\theta}}|{\bf y}_t) = \mathbb{E}_{p({\bf w}|D)}[\tilde{p}(\hat{\boldsymbol{\theta}}|{\bf y}_t, {\bf w})]$ is the expectation on unseen data. Setting ${\bf y}_t = {\bf y}_0$, for $\hat{\boldsymbol{\theta}} \approx \boldsymbol{\theta}$ the proposal posterior predictive distribution will be equal to $\tilde{p}(\boldsymbol{\theta}|{\bf y}_0)$ as $N \rightarrow \infty$, which is the target parameter proposal posterior conditioned on a particular observation. 
 %Additionally, when the probabilistic model $p(\boldsymbol{\theta}|{\bf y},{\bf w})$ is a regression model, it also will yield $\hat{\boldsymbol{\theta}} = \boldsymbol{\theta}$. However, 
 As we demonstrate next, we will approximate the parameters $\boldsymbol{\theta}$ through discrete categories making ${\tilde p}(\boldsymbol{\theta}|{\bf y},{\bf w})$ a categorical distribution thereby transforming the regression problem into a classification problem. 

\subsection{Probabilistic regression via classification}
Let $\tilde{p}(\boldsymbol{\theta}'|{\bf y},{\bf w})$ be represented as a categorical distribution using cross-entropy and softmax activation in a NN classifier denoted by $F_{\tilde{p}(\boldsymbol{\theta}'|{\bf y},{\bf w})}$.  We will transform the continuous problem by discretizing $\boldsymbol{\theta} \in \mathbb{R}$ into $\boldsymbol{\theta}'\in K$ by $B$ equal width bins per dimension (please see Section~\ref{sec:discussion} for a discussion on binning strategies), i.e., the predictive distribution will provide the probability of belonging to a particular bin $K \in {1,2,...,B}$. By placing a multivariate Gaussian on $p({\bf w})$ we will fit $p({\bf w}|D)$ using Bayes by backprop, subsequently yielding the BNN classifier  $F_{p({\bf w}|D)}$. To estimate the proposal posterior predictive distribution conditioned on our target observation, we sample from $F_{p({\bf w}|D)}$ using a MC sampling scheme,   
\begin{equation}
    \tilde{p}(\boldsymbol{\theta}'|{\bf y}_0) \approx \frac{1}{N}\sum_{i=1}^N F_{p({\bf w}|D)}({\bf y}_0, {\bf w}_i).
\end{equation}

To enable sequential adaptive sampling during the inference task, we will train $G$ subsequent classifiers $F^g_{p({\bf w}|D)}, \, g = 1, \dots, G$, where the bins are updated by sampling from the proposal posterior predictive distribution. Each iteration employs the density from the previous generation as the prior for sampling new training data. The BNN generates amortized proposal posterior estimates. However, when conditioned on the observation, it allows for adaptive refinement in each iteration. To be able to simulate new samples, we need to transform $\boldsymbol{\theta}'$ back to $\boldsymbol{\theta}$. This is achieved by transforming each bin $K$ into a uniform distribution,

\begin{equation}
\begin{aligned}
    \{\boldsymbol{\theta}'\}_{i=1}^N &\sim \tilde{p}(\boldsymbol{\theta}'|{\bf y}_0) \\
    \{\boldsymbol{\theta}\}_{i=1}^{N} &= \{\mathbb{U}^K({\boldsymbol{\theta}'}_i^{\text{\emph{left}}},{\boldsymbol{\theta}'}_i^{\text{\emph{right}}})\}_{i=1}^N,
\end{aligned}
\end{equation}

where \emph{left} and \emph{right} are the boundaries of the bin intervals. The adaptive sampling-driven BNN is depicted in Figure~\ref{fig:pipeline} and detailed in Algorithm~\ref{alg:posterior}.

\begin{figure}[t]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/pipeline_v3.pdf}}
\caption{Adaptive sampling-driven BNNs. (A) Parameter proposals are sampled from a prior and simulated via the model. (B) Regression via classification. Continuous proposals are discretized into a categorical distribution with classes being the bins. A BNN (having prior weights) is then trained on data. (C) The posterior predictive conditioned on observation $y_0$ can be estimated via MC sampling, and is subsequently used in an iterative procedure, replacing the previous prior. Note that a transformation is applied to the categorical distribution to transform proposals back to continuous.}
\label{fig:pipeline}
\end{center}
\vskip 0.2in
\end{figure}

\subsection{Exploitation of bins}
%With large priors $p(\boldsymbol{\theta})$, finding the subset $p(\boldsymbol{\theta}|{\bf y}_0)$ can be a slow process.% 
We aim to exploit the uncertainty estimates provided by the proposal posterior predictive distribution to enable faster convergence. A key reason why the process can be slow is that low probability weights linger between iteration updates. We exploit the proposal posterior predictive distribution by using a threshold value, $\delta$. Given the categorical distribution, we set the classes with probability weight below the threshold to zero. The non-zero probabilities are then normalized, constructing a less skewed proposal distribution. Between iterations, $\delta_i$ can be chosen adaptively or kept constant throughout. We recognize that the sequence of distinct $\delta_i$ holds a similarity to the discrepancy thresholds ($\epsilon_i$) used by ABC-SMC~\cite{sisson2007sequential, del2012adaptive}.
% ** if not adaptive **
The potential adaptive nature is briefly discussed in Section~\ref{sec:discussion}.
%% *** if we decide on using adaptive ***
%To slow the exploitation, we employ an $\epsilon$ that converges to zero as the rounds increase.



\subsection{Dealing with the curse of dimensionality}
For the single-parameter problem, the structure of the classification problem is straightforward: one classifier for one parameter. However, for multiple parameter dimensionality, we construct one classifier for each two-dimensional covariance combination: $(\boldsymbol{\theta}_i, \boldsymbol{\theta}_j)$, implying that for a model of $k$ parameters, we would require $\frac{k!}{(k-2)!2!}$ classifiers. One sees that the number of classifiers quickly increases; however, they lend themselves towards an embarrassingly parallel workflow since they are independent, and use the same simulation data. Further, the adaptation of the proposals is done per covariance dimension and then marginalized, resulting in the $k$-dimensional parameter proposals.


%The LFI problem is solved by finding the inverse map $F(y)\mapsto \theta$. Consider the approximation $F_w(y)$ produced by an embedded net with weights, $w$. By fitting the inverse mapping function $F_w(y)$ using simulated data $\mathcal{D} = \{(y_i,\theta_i)\}_1^N$, a loss function $\mathcal{L}(\theta_i, \hat{\theta_i})$ can be optimized w.r.t. $w$ between predicted and true parameters; the weights $w$ should ultimately incorporate the discrepancy between samples.
%Our probabilistic approach allows incorporating epistemic uncertainty by employing Bayesian embedding nets (BNN) \cite{mackay1992practical}, which learn distributions of $q(w|\mathcal{D})$ instead of a point estimate $\hat{\theta}$ by introducing a prior $p(w)$ on the weights. Approximating the predictive posterior $p(y_0 |\mathcal{D})$ by Monte Carlo integration gives an interpretation of $\hat{p}(\theta | y_0)$, namely approximating the integral $ p(y_0 |\mathcal{D}))  = \int p(y_0 |\mathcal{D}),w)q(w|\mathcal{D}))dw$
%by a sum of independent samples $ p(y_0 |\mathcal{D})) \approx \frac{1}{M}\sum_{i=1}^M p(y_0 | w_i)$
%where $w_i \sim q(w | \mathcal{D}))$. 
%A connection between the NN and the BNN is that NN's weights can be seen as maximum likelihood estimates or maximum a posteriori estimates, when using regularization. %\cite{}? could use, https://arxiv.org/pdf/2007.06823.pdf but not sure how close they are to be published results.

%\begin{equation}
%    \begin{aligned}
%        p(y_0 |\mathcal{D}))  = \int p(y_0 |\mathcal{D}),w)q(w|\mathcal{D}))dw\\
%         \approx \frac{1}{M}\sum_{i=1}^M p(y_0 | w_i)\\
%         \approx \hat{p}(\theta|y_0)\\
%         \text{where } w_i \sim q(w | \mathcal{D}))
%    \end{aligned}
%\end{equation}


% The classification problem
%\subsection{Accelerating posterior estimation using classification}
%To achieve robust results (\textcolor{red}{FW: this is not enough motivation for using classification}), around the sought after posterior distribution $\hat{p}(\theta|y_0)$, we consider a sequence of discrete mappings from $y \mapsto \theta$ with increasing resolutions. Therefore, we solve finding the continuous map $F$ around $y_0$ by a series of approximate mappings, i.e., $F = f_1 \circ f_2 \circ \dots \circ f_G$. 
%The discrete map $f_k$ considers binned continuous data, meaning, finding $f_k$ is a classification problem for a categorical distribution. We condition each iterative map on $y_0$ to determine the area of interest in the prior, i.e., $f_{k+1}$ follows $f_k$, and $P_{f_k}(\theta|Y_0)$ gives the samples of interest. 
% The final inference step
%A key feature inherent of BNN is that the map $F$ direcred}tly gives the probability $p(\theta|y)$, and given the observation $y_0$, we get $\hat{p}(\theta|y_0)$ without any further steps.


%\section{Classification for LFI using Bayesian neural networks}
%Our approach's contribution is that the BNN allows for uncertainty estimates directly while on the classification step, instead of using the accuracy as a pseudo-measure for the discrepancy between the observation and simulation. We construct the classes by grouping continuous-data into a fixed number of bins. As the number of bins approach the number of samples, we approach the continuous, regression case.  

%The posterior density estimation by BNN is generated iteratively by a sequence of generation densities.  Each iteration employs the previous generation density as it's prior for sampling new training data. The BNN generates amortized posterior estimates. However, when conditioned on the observation, it allows for adaptive refinement in each iteration.

%Denote the trained BNN as $F(y|w)$ with weights $w$, the posterior distribution, approximated by the BNN, $q_{F(y,w)}$. We generate $q_F$ by Monte Carlo-sampling the network, and retrieve a categorical distribution over the binned data.


\begin{algorithm}[]
   \caption{Iterative BNN proposal posterior estimation }
   \label{alg:posterior}
\begin{algorithmic}
   \STATE {\bfseries Input:} model simulator $M({\bf y}|\boldsymbol{\theta})$, 
   observations ${\bf y}_0$, Bayesian neural network $F_{p({\bf w}|D)}$, 
   number of generations $G$, number of simulations per generation $N$,
   prior distribution $p(\boldsymbol{\theta})$, number of bins $B$.

   \STATE $p_0 := p$  
   \FOR{$g=1$ {\bfseries to} $G$}
        %\FOR{$n=1$ {\bfseries to} $N$} % training
            \STATE Sample $\boldsymbol{\theta}_N = \{\boldsymbol{\theta}_i\}_{i=1}^N$ from $p_{g-1}(\boldsymbol{\theta})$
            \STATE Sample ${\bf y}_N = \{{\bf y}_i\}_{i=1}^N$ from $M({\bf y}|\boldsymbol{\theta}_i)$
       % \ENDFOR
        \STATE Group $\boldsymbol{\theta}_N$ into K bins $\boldsymbol{\theta}'_N, \, K= 1:B$
        \STATE Train $F^g_{p({\bf w}|D_g)}$ on $D_g = \{{\bf y}_N, \boldsymbol{\theta}'_N\}$
        \STATE Estimate $\tilde{p}_{g}(\boldsymbol{\theta}'_N|{\bf y}_0) \leftarrow \mathbb{E}[F^g_{p({\bf w}|D_g)}({\bf y}_0,{\bf w}_N)]$
        \STATE Transform $p_g(\boldsymbol{\theta}) \leftarrow \mathbb{U}^K({\tilde{p}}_{g}(\boldsymbol{\theta}'_N|{\bf y}_0))$
        %\STATE \textcolor{red}{FW: Not sure what notation to use for the transform}
    \ENDFOR
    \STATE {\bfseries return} $p_g(\boldsymbol{\theta})$
\end{algorithmic}
\end{algorithm}

%\subsection{Complexity}
%\textcolor{red}{FW: Can we say something more about the complexity of the approach more than the number of weights? E.g simulations $>>$ training time? Also curse of dim, several classifiers are embarrassingly parallel and the number of simulations does not increase. Very important that the reader understand this.} 

%\subsection{Design of the BNN}
%\textcolor{red}{FW: I think this part should go into experiments as a particular case of the BNN tailored for dynamic models}
%In this work we have implemented the BNN as Convolutional NN, henceforth called BCNN. However, we would to point out that many other types of architectures can be used. Out motivation behind this choice is that CNNs have shown great performace in deducting inormation from time-series data \cite{sadouk2019cnn, aakesson2020convolutional}, and not only for images. To turn the layers of a CNN into probabilitic layes we have used the flipout \cite{wen2018flipout} layes supported in Tensorflow \cite{tensorflow2015-whitepaper} probability package. 

%\subsection{Adaptive sampling}


\section{Experiments}
\label{sec:experiments}
%\textcolor{red}{FW: missing transition into CNN (BCNN), have not decided if it should be introduced in section \ref{sec:method} or as a particular case of a BNN in section \ref{sec:experiments}}

To test our approach we have implemented a convolutional BNN (we denoted this particular implementation BCNN). However, we note that our approach is not restricted to these types of architectures.  Our motivation behind choosing a convolutional architecture is that they have demonstrated excellent performance in identifying intricate patterns in complex, high-dimensional time-series data \cite{dinev2018dynamic, sadouk2019cnn, aakesson2020convolutional} as well as images~\cite{APT_2019}. For further information on the implementation we refer the reader to supplementary materials.  


We explore the performance of the BCNN on two benchmark examples - the moving averages model (MA) and the discrete-state continuous-time Markov chain implementation of the Lotka-Volterra (LV) \cite{lotka}. Together with the resulting proposal posterior we recover using the BCNN, we also compare the proposal posterior estimation quality to posterior estimates obtained using ABC-SMC and SNPE-C with the MAF as the neural density estimator.

The SBI package \cite{tejero2020sbi} implements SNPE-C, and is used for the experiments in this paper. To keep the comparison fair between SNPE-C and the BCNN, we designed the embedded nets to be as similar as possible between the two approaches, resulting in having approximately the same number of weights in the two models.  
 
All of the experiments were conducted on a machine consisting of a 12-core Intel Xeon E5-1650 v3 processor operating at 3.5 GHz, 64 GB RAM, running Fedora core 32 Linux operating system. Even though GPU-acceleration would speed up NN training substantially, in the considered setting we assume that simulations are significantly more expensive than training time. This is the case for the discrete and stochastic LV model, less so for MA.   


\subsection{Moving average model}%- Noisy, high data dimensionality
% \begin{equation}
%     X_t = \epsilon_t + \theta_1\epsilon_{t-1} + \theta_2\epsilon_{t-2},
%     \epsilon_t \sim N(0,1) 
% \end{equation}
The moving average (MA) model is a popular benchmark problem in the ABC setting \cite{marin2012approximate}. The MA model of order $q$, MA($q$) is defined for observations $X_1, \dots, X_p$ as \cite{wiqvist2019partially,aakesson2020convolutional},
\begin{equation}
    X_j = Z_j + \theta_1 Z_{j-1} + \theta_2 Z_{j-2} + \dots + \theta_q Z_{j-q},
\end{equation}
where $j=1,\dots,p$, and $Z_j$ represents latent white noise. For the experiments in this paper, we consider $q=2$ and $Z_j \sim N(0,1)$ \cite{wiqvist2019partially}. The MA(2) model is identifiable in the following triangular region,
\begin{equation}
    \theta_1 \in [-2,2], \theta_2 \in [-1, 1], \theta_2 \pm \theta_1 \geq -1.
\end{equation}
In order to ease implementation and compatibility between the compared approaches, a rectangular uniform prior encompassing the triangular region was used in the experiments. We note here that the estimated (proposal) posterior was always within the triangular region, therefore no identifiability concerns were encountered due to a rectangular prior.

%\textcolor{red}{TODO: However we use a uniform prior...how do we motivate this choice? Both SNPE-C and ABC-SMC need a pdf of the prior. BCNN don't. Further the SBI package don't have an API supporting user-defined priors, which is baaaad}

\subsubsection{Experimental Setup}
In the experiment we conduct a total of $G=6$ iterations (rounds) of refining the (proposal) posterior estimate for 10 individual seeds. The total amount of parameter proposals in each round is set to $N=3000$ and the number of bins for BCNN is set to $B=4$ per parameter dimension, resulting in a total of 16 bins. The exploitation threshold is set to $\delta = 0.05$. We simulate the MA(2) model with 100 equidistant time points.
For the ABC-SMC method, we use auto-correlation coefficients of lag 1 and 2, and variance as summary statistics. They are non-sufficient, but are chosen as a representation of a typical choice made by a practitioner in absence of much prior knowledge. Euclidean distance was used as the discrepancy measure.

\subsubsection{Results}
Here we demonstrate the perfomance of the proposed BCNN approach and compare against SNPE-C and ABC-SMC. The model has a superficial posterior, but the map $\boldsymbol{\theta} \mapsto {\bf y}$ is ill-discriminated, i.e., considerable variation in $\boldsymbol \theta$ provides little feedback in ${\bf y}$.  %Figure~\ref{fig:grid_MA2} A shows a qualitative evaluation of two sample executions(seed) of BCNN and SNPE-C. We observed that the BCNN is more robust in estimating the posterior as compared to SNPE-C, which we further demonstrate in Figure~\ref{fig:emd_MA2} by calculating the Earth Moving Distance (EMD, also referred as Wasserstein distance) and Maximum Mean Discrapancy (MMD) between estimated posteriors and the exact posterior. Finally, we also include ABC-SMC for reference; however, this method requires a substantial increase of simulations per round for convergence.  

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_grid_4_25x4_25_fontsize8.pdf}}
\caption{MA(2) per round proposal posterior estimation for BCNN, compared to approximated posteriors obtained using SNPE-C, and ABC-SMC, together with the ground truth - a MCMC generated posterior. Each round includes $N=3000$ parameter proposals, except for ABC-SMC which require many more. BCNN and SNPE-C includes two different random seeds (1) and (2). Final column to the right shows a Gaussian kernel density estimate of the final round posterior per method.}
\label{fig:grid_MA2}
\end{center}
\vskip 0.2in
\end{figure}

Figure~\ref{fig:grid_MA2} provides qualitative insights on the shapes of the estimated proposal posterior obtained using the proposed approach, and posterior distributions obtained using the other approaches. Figure~\ref{fig:emd_MA2} depicts the log mean earth mover distance (EMD)~\cite{levina2001earth} and the mean of maximum mean discrepancy (MMD) \cite{Scholkopf_MMD} between the estimated distributions and the exact posterior. We see similar convergence patterns between BCNN and SNPE-C for both the EMD and MMD over subsequent rounds and different seeds, with the BCNN having a slight edge. This is despite the BCNN targeting a proposal posterior rather than the true posterior with a correction step. It can also be observed that the BCNN is highly robust over different seeds with very little variance in performance.
Another interesting observation from Figure~\ref{fig:grid_MA2} is the improvement or refinement extracted by each approach over subsequent rounds. It can be seen that BCNN(1) and BCNN(2) are able to substantially improve the quality of the estimated distributions between rounds 2-4 and 4-6. The ability of the BCNN architecture to greatly improve over successive rounds can be attributed to the combination of uncertainty estimates from the BCNN model leveraged by the adaptive sampling algorithm to identify regions most likely to contain the true posterior.

%The lack of variability between seeds, the error bars in the figure showcase the robust properties of the BCNN method.

\begin{figure}[ht]

\includegraphics[width=\columnwidth]{img/ma2_emd_mmd_4_25x2_fontscale08.pdf}
\caption{Target posterior error estimates, log mean EMD (left) and Mean MMD (right) for BCNN, SNPE-C and ABC-SMC per round, with variability $\pm$ 1 std, recovered from ten independent seeds. The BCNN have a slight edge over SNPE-C, especially when it comes to the variability. Note that on average ABC-SMC will require more simulations to converge than the other two, see bottom row in Figure~\ref{fig:grid_MA2}.}
\label{fig:emd_MA2}

\end{figure}

\subsection{Lotka-Volterra predator-prey model}
The Lotka-Volterra predator-prey dynamics model is a popular parameter estimation benchmark problem \cite{aakesson2020convolutional, greenberg2019automatic}. The model is implemented as a stochastic Markov jump process \cite{prangle2017adapting} simulated using the stochastic simulation algorithm (SSA) \cite{gillespie1977exact}. The model consists of three events - prey reproduction, predation and predator death. The model is described by the events:% $\mathcal{X}_1 \longrightarrow 2\mathcal{X}_1$, $ \mathcal{X}_1 + \mathcal{X}_2 \longrightarrow 2\mathcal{X}_2$ and $ \mathcal{X}_2 \longrightarrow \phi$.
\begin{equation}
 \begin{aligned}
     \mathcal{X}_1 &\xrightarrow{\theta_1} 2\mathcal{X}_1,\\
     \mathcal{X}_1 + \mathcal{X}_2 &\xrightarrow{\theta_2} 2\mathcal{X}_2,\\
     \mathcal{X}_2 &\xrightarrow{\theta_3} \emptyset.
 \end{aligned}
\end{equation}
The parameters ${\boldsymbol{\theta}} = \{ \theta_1, \theta_2, \theta_3\}$ control the events described above, and have corresponding rates $\theta_1 \mathcal{X}_1, \theta_2 \mathcal{X}_1 \mathcal{X}_2, \theta_3 \mathcal{X}_2$ respectively. 
\subsubsection{Experimental Setup}
% K = 5
% G = 8
% N = 1000
% \delta = 0.05
We study the inference using a total of $G=8$ rounds of refining the target distribution, with five repetitions using different random number seeds, setting the total amount of parameter proposals in each round to  $N=1000$. Further, we define the number of bins to $B=5$ per dimension and the threshold to be $\delta = 0.05$.
The LV model data consists of 51 observations in the time span $[0 \text{ h},50 \text{ h}]$. We construct the model and the corresponding simulator using \textit{GillesPy2}\footnote{https://github.com/StochSS/GillesPy2} \cite{gillespy_2017}. 
Finally, the initial conditions include $\mathcal{X}_1=50$ and $\mathcal{X}_2=100$, and the actual parameters to be inferred are $[\log(1.0), \log(0.005), \log(1.0)]$. The prior we apply per dimension is $\mathbb{U}(\log(0.002), \log(2))$. We include the ABC-SMC estimate, because of the lack of a true posterior solution, and the LV model is a common benchmark problem in the ABC literature, i.e., the solution it produces is well accepted \cite{SNPE_A_Papamakarios,owen2015scalable}.
For the ABC-SMC method, we use no summary statistics, i.e., the data itself is directly compared, and a Euclidean distance as the discrepancy measure. The tolerance thresholds ($\epsilon$'s) for SMC-ABC are selected using a relative scheme, where the $20$-th percentile value of the ABC distances from the previous round is selected as $\epsilon$.
\subsubsection{Results}
% MSE
Compared to the MA(2) model, we do not have an analytical posterior to compare to, thus we cannot refer to the EMD of the method. However, we solve the problem with known truth (the parameters to infer) and consider the mean square error (MSE) for computed target posteriors. The reasoning is that MSE holds information both about the bias and the variance of the distribution. But with the known limitation of not considering any higher moments. In Figure~\ref{fig:MSE_LV}, we evaluate the said MSE of the BCNN approach together with SNPE-C and ABC-SMC. The BCNN performs on par with SNPE-C, without any loss of posterior concentration; see post round 5 how the standard deviation increases per generation of SNPE-C. Exploring what might have caused the increase in standard deviation, we refer to Figure~\ref{fig:grid_LV}, which presents snapshots of a sequence of rounds from one run with the different methods. SNPE-C encounters an instability after round 4. From the MSE, we note that SNPE-C is mostly, not always, stable compared to the empirically robust BCNN.
Finally, we recognize that the initially poor performance of the ABC-SMC method can be attributed to the lack of adequately tuned summary statistics.

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_mse}}
\caption{The lines with error bars show the mean MSE with variability $\pm1$ std, recovered from five seeds with eight rounds for SNPE-C, BCNN, and ABC-SMC. All methods converge to similar MSE at the final round, with low variability. Notable is the initial increase in MSE for ABC-SMC; from Figure~\ref{fig:grid_LV} it can be inferred that the approximated posterior (proposal posterior in case of BCNN) is bi-modal before finding a suitable discrepancy threshold.}
\label{fig:MSE_LV}
\end{center}
\vskip 0.2in
\end{figure}

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_dens}}
\caption{Snapshots of the proposal posterior estimate for the LV problem using BCNN, and posterior estimates using SNPE-C, and ABC-SMC. Each column is a different round: 2, 4, and the final 8, and each row gives the three parameters combinations possible of $\theta_1$, $\theta_2$, and $\theta_3$. The true parameter value is indicated by the orange cross. The prior covers the entire box uniformly. For the ABC-SMC the cumulative number of trials at round 2, 4 and 8 are $[8000, 19000, 127000]$, compared to $[2000, 4000, 8000]$ for BCNN and SNPE-C.}
\label{fig:grid_LV}
\end{center}
\vskip 0.2in
\end{figure}


%\subsection{Example 3?}
% placeholder
\section{Discussion}
\label{sec:discussion}
% Gotchas and pitfalls
%\textcolor{red}{PS: Can we mention some pitfalls, something to be careful about and pointers on how to get the best out of our approach?}
% - by using classification we cannot re-use the trained network, we have to start anew each round. Meaning that the information gained from each trained network must be leveraged to it's fullest when constructing the proposal distribution.
% - number of parameter dimensions

The results presented in the previous section demonstrate that the proposed approach is comparable to SNPE-C, which is a well-established method for approximating flexible posterior densities using NNs. Notably, the comparable performance has been achieved in spite of a lack of correction of the proposal posterior in the case of the BNN. The uncertainty estimates and a simplification strategy of accomplishing regression via classification using the BNN allows the adaptive sampling algorithm to consistently drive the search for the true posterior in the right direction. This can be seen in the very low standard deviations presented in Figures~\ref{fig:emd_MA2} and \ref{fig:MSE_LV}. 
However, an important observation is that SNPE-C does not utilize any exploitation similar to $\delta$ in the proposed approach  (see Section~\ref{sec:method}). Effectively, the deviations in performance are reduced using this exploitation. 

However, incorporating a similar exploitation strategy used in SNPE methods may not alone resolve the issue of occasional, substantially inaccurate posterior estimates, such as SNPE-C(2) in Figure \ref{fig:grid_MA2}. The stability and robustness of the proposed approach stems from the Bayesian setting considered in this work. Further, the BNN is specifically tailored for models with complex data distributions, such as discrete stochastic and dynamical models. 
%The BNN functions as both  an embedding network to extract summary statistics and as a proposal posterior estimator. 
% However, the use of categorical distributions as proposal posterior estimators is limited to fairly simplistic posteriors. For flexible posterior estimators we recommend SNPE-C, but to utilize the robustness and the exploitative properties of the proposed approach one might consider using both approaches in conjunction with each other. For example, using the BNN as a pre-processing step to robustly define a prior distribution, and subsequently passing the proposal posterior estimate $\tilde{p}(\boldsymbol{\theta}'|y_0)$ as the prior for SNPE-C. This approach is outlined in Algorithm~\ref{alg:proposed}.

% \begin{algorithm}[]
%   \caption{Proposed: BNN in conjuction with SNPE-C}
%   \label{alg:proposed}
% \begin{algorithmic}
%   \STATE Do Algorithm~\ref{alg:posterior} to get $\tilde{p}_G(\boldsymbol{\theta})$
%   \STATE Do SNPE-C with $\tilde{p}_G(\boldsymbol{\theta})$ as prior
% \end{algorithmic}
% \end{algorithm}




% Significance of uncertainty estimates provided by the B(C)NN
%The results presented in the previous section underscore the importance of accounting for embedding model uncertainty towards the robustness of resulting LFI quality. The uncertainty estimates provided by the BNN allow the adaptive sampling algorithm to consistently drive the search for the true posterior in the right direction. This can be seen in the very low standard deviation of parameter inference error shown in Figures~\ref{fig:emd_MA2} and \ref{fig:MSE_LV}.

% Significance of adaptive sampling and categorical classification
%Further, adaptive sampling also enables consistent and stable refinement of the estimated posterior over successive rounds of algorithm execution. \textcolor{red}{PS: @Fredrik: pls put in some text here connecting how adaptive sampling and categorical classification is responsible for the refinement. Further, would be good to contrast with SNPE and talk about our advantages}

% Dynamical models
%\textcolor{red}{PS: why is our approach well-suited for dynamical models}
% + find "optimal" summaries, which is a hard task for time-series/dynamical models. Able to handle high dimension data well. 
\subsection{Future work}
% Future work
%\textcolor{red}{PS: Future work, explore more sophisticated sampling algorithms and the interplay between exploration and exploitation}

The convergence plots in Figure~\ref{fig:emd_MA2} and \ref{fig:MSE_LV} point towards an opportunity  for early stopping when the proposal posterior estimate no longer receives significant updates. A possibility could be to measure the highest density region (HDR)~\cite{HDR}, or EMD, change between proposal posterior estimates, and stop if the difference is below a threshold value. Using this could substantially reduce the number of simulations involved and remove the requirement of specifying the amount of rounds $G$.

Choosing the optimal number of bins $B$ and number of proposals $N$ in each bin (support) is currently not trivial. A sufficient support for dynamical models is highly dependent on the response of simulation outputs to perturbations in the parameter space. For the MA(2) model, a larger $N$ per round was required, as well as fewer bins compared to the settings for the LV problem. This was owing to the MA(2) model mapping being ill-discriminated in nature as described earlier. Thus, for the BNN classifier to sufficiently learn discriminating features, one would require more data. Further, currently the bin width is fixed and does not consider the distribution of proposals. Exploring variable bin widths based on the rate of change of proposal distributions over successive rounds could potentially improve convergence speed, and also balance the support for bins more evenly. As a remark on the choice of number of bins, in \ref{sec:hyperMA2} and \ref{sec:hyperLV} we present the result from our empirical tests, which encourages further studies.

The procedure of assigning a continuous $\theta_i$ into a bin does not take discretization uncertainty into account (e.g., when $\theta$ is close to the egde of two bins), which can lead to inaccurate training. The aforementioned problems will be considered as future work and will involve adaptive bin selection based on the discriminative ability of the BNN. Further, bin uncertainties could potentially be tackled using fuzzy set theory and other bin strategies used in regression via classification.

The hyperparameter $\delta$ (see Section~\ref{sec:method}) that controls exploitation is currently considered to be fixed. However, an adaptive $\delta$ dependent on the current round can potentially improve convergence, e.g., an exponentially decaying $\delta$. The similarity to the threshold one use in ABC-SMC, aligns the adaptivity of $\delta$ well with strategies already in practice. In \ref{sec:hyperMA2} and \ref{sec:hyperLV} we give additional test results on the subject. 

A regression-based approach as opposed to the classification based approach presented in this paper will be explored. We will also work towards integrating the proposal correction process as part of the algorithm in order to estimate the true posterior.

 %\textcolor{red}{FW: please elaborte, I don't understand}
 %One could potentially consider making it an adaptive procedure, dependent on how well the BNN resolves the previous step; if one observes a low effective sample size, then decrease the number of bins and vice versa. Additionally, currently, the bins are of equal width per dimension; by leveraging information from the proposal distribution, one could explore more informative bin widths, potentially bridging learning between rounds.


%discussion about early stopping, constructing bins, number bins and suggested exploitation. (the number of weights in BNN increase as number of bins increase, fuzzy bins?



\section{Conclusion}
\label{sec:conclusion}
This paper presented a novel Bayesian embedded neural network based approach for learning summary statistics for parameter inference of dynamical simulation models. The proposed approach leverages Bayesian neural networks for learning high-quality summary statistics as the estimated proposal posterior while also incorporating epistemic uncertainties. An adaptive sampling algorithm utilizes the estimated proposal posterior and uncertainty estimates from the Bayesian neural network to iteratively shrink the prior search space for the actual posterior distribution. An efficient classification-based density estimation approach further approximates the proposal posterior. The proposal posteriors obtained using the proposed approach are compared to approximated posteriors obtained using a well-established sequential neural posterior estimation method and the popular approximation Bayesian computation sequential Monte Carlo likelihood-free inference methods. Experiments performed on benchmark test problems show that the proposed approach is consistently more robust under various situations compared to existing methods while delivering comparable parameter inference quality.

\bibliography{references}
\bibliographystyle{icml2021}

\begin{appendix}
\onecolumn
%\icmltitle{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\icmltitle{Robust and integrative Bayesian neural networks\\ for likelihood-free parameter inference\\
Supplementary Material}

\section{Extended Experiments for the MA(2) Test Problem:}
We investigate the effect of varying random seeds, and a comparison of execution times for the proposed approach against SNPE-C. We also explore the effect of varying the hyperparameters involved in the proposed BCNN-based approach.
\subsection{Qualitative evaluation with additional seeds}

\begin{figure*}[ht]
\begin{center}
\centerline{\includegraphics[width=\textwidth]{img/ma2_suppl_grid.pdf}}
\caption{Ten individual seeds for estimating posterior using SNPE-C (A) and proposal posteriors using BCNN (B). }
\label{fig:ma2_grid_suppl}
\end{center}
\end{figure*}


\twocolumn
In order to investigate the robustness of the proposed approach, ten distinct random seeds were used in the parameter inference task for the BNN (BCNN) and SNPE-C based methods. Figure~\ref{fig:ma2_grid_suppl} depicts the results with each row representing a seed, and each column representing a round in the sequential parameter inference process. The random walk Metropolis-Hastings MCMC method was used to calculate the exact posterior for the purpose of comparisons with BCNN and SPNE-C. 

It can be observed that the proposed BCNN-based approach is consistent across different seeds, while SNPE-C shows susceptibility in three cases (seeds 4, 5 and 9). It can also be seen that SNPE-C typically starts with a tighter estimated posterior in the first round, but relative gains over subsequent rounds are small. In comparison, BCNN is slower in the initial stages, with larger estimated proposal posteriors in the first round. However, relative gains over successive rounds are more significant with substantial refinement being achieved. This can be attributed to the exploitation offered by the binning-based classification approach involved in parameter inference using BCNN. The final distribution in the sixth round is more concentrated for the BCNN as compared to SNPE-C. 

\subsection{Embedding Network architectures}
\label{sec:arch_ma2}
The BCNN was implemented using the TensorFlow library (Keras and tensorflow probability), while the SBI package (SNPE-C) makes use of PyTorch. The architectures are shown in Table \ref{tab:ma2}.
\begin{table}[ht]
\caption{Architecture for the BCNN (top) and SNPE-C (bottom); NDE refers to neural density estimator.}
\label{tab:ma2}
%\vskip 0.15in
\begin{center}
%\begin{small}
%\begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 100, 1)]&0\\
Conv1DFlipout (ReLU) & (None, 100, 6)  &66\\
MaxPooling1D    & (None, 10, 6)&0\\
Flatten    & (None, 60)&0\\
DenseFlipout (Softmax)     &(None, 16)&1936\\
\midrule
Total weights &&2002

\end{tabular}
%\end{sc}
%\end{small}
%\end{center}

%\caption{Architecture SNPE-C, icluding embedded summary statistics network and neural density estimator (NDE) with 10 hidden units and 2 transforms}
% \label{sample-table}
% \vskip 0.15in
% \begin{center}
% \begin{small}
% \begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 100, 1)]&0\\
Conv1d (ReLU) &(None, 100, 6)  &36\\
MaxPooling1d    &(None, 10, 6)&0\\
view (Flatten)    & (None, 60)&0\\
Linear (ReLU)     &(None, 16)&488\\
NDE (MAF) &10 hidden, 2 transform&970\\
\midrule
Total weights &&1494

\end{tabular}
%\end{sc}
%\end{small}
\end{center}
%\vskip -0.1in
\end{table}

\subsection{Hyperparameters of the BNN}
\label{sec:hyperMA2}
Here we experimented with different hyperparameters for the BNN (BCNN), including different number of bins (classes) in Figure~\ref{fig:ma2_bins}, and different exploitation strategies using the threshold $\delta$ in Figure~\ref{fig:ma2_thresh}. For MA(2), four bins per parameter (16 classes) is the best performing setting, however using five bins (25 classes) results in similar performance. For the exploitation threshold $\delta$, we observe the best overall performance using $\delta=0.05$ when observing log mean EMD. For mean MMD both show similar results. Additionally, a proposed strategy of using an exponentially decaying threshold, i.e., reducing the amount of exploitation per round, performs on a level in between $\delta = 0.05$ and $\delta = 0.0$. 
\begin{figure}[ht]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_bins__4_25x2_fontscale08.pdf}}
\caption{Log mean EMD and mean MMD for BCNN with different number of classes corresponding to the number of bins.}
\label{fig:ma2_bins}
\end{center}
\end{figure}

\begin{figure}[ht]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_thresh__4_25x2_fontscale08.pdf}}
\caption{Log mean EMD and mean MMD for BCNN for different threshold $\delta$. $\delta = 0.0$ refer to no threshold being used (i.e no exploitation). $\delta = 0.05$ is the strategy used in the main article. $\delta = e$ is an exponentially decaying threshold.}
\label{fig:ma2_thresh}
\end{center}
\end{figure}

\subsection{Execution time}
Figure~\ref{fig:ma2_time} shows the time difference for the inference task between the two implementations used in this study. Note that the BCNN used for MA(2) is one joint model (2D bins), however for the LV model several co-variate BCNN models are trained.

It is important to note that differences in software implementations (TensorFlow versus PyTorch) and approaches to parallelization make direct empirical comparisons challenging. Further, factors such as the early stopping criterion will also have an effect on the result.

%Note that this is by no means a final evaluation of the computational complexity between BCNN and SNPE-C. Both have different implementations and are using various parallel and neural network packages, further factors such as early stopping criterion will also have an effect on the result. However, Figure~\ref{fig:ma2_time} shows the time difference for the inference task between the two implementations used in this study. Observe that the BCNN used for MA(2) is one joint model (2D bins), however for the LV model several co-variate BCNN models are trained. 

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_time__4_25x2_fontscale08.pdf}}
\caption{(left) Time (minutes) per round between BCNN and SNPE-C using MAF. (right) Cumulative time (minutes). Both include simulation time per round (3000 simulations/round).  }
\label{fig:ma2_time}
\end{center}
\end{figure}

%\subsection{Various neural density estimators in SNPE-C}
%\begin{figure}[ht]
%\vskip 0.2in
%\begin{center}
%\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_NDE__4_25x2_fontscale08.pdf}}
%\caption{Text}
%\label{fig:ma2_time}
%\end{center}
%\vskip 0.2in
%\end{figure}




\section{Extended Experiments for the Lotka-Volterra (LV) Test Problem:}
The structure of extended experiments for the LV test problem follows the MA(2) setup in the text above. 
%We investigate the effect of varying random seeds, a comparison of execution times for the proposed approach against SNPE-C. We also explore the effect of varying the hyperparameters involved in the proposed BCNN-based approach.

\subsection{Qualitative evaluation with additional seeds}
%To investigate the robustness in the LFI-methods, we repeat the experiments multiple times, with five different seeds. 
The robustness of the three approaches - BCNN, SNPE-C and ABC-SMC against five different seeds is now investigated for the Lotka-Volterra predator-prey model.
In Figure~\ref{fig:lv_grid}, we show the snapshots and the final two-dimensional marginal distributions for the five independent runs. Comparison of the three approaches shows that the BCNN approach is relatively robust, while the ABC-SMC method converges slowly. It can also be seen that SNPE-C often converges to a different distribution shape as compared to BCNN and ABC-SMC. 



% \begin{figure}[ht]
% \vskip 0.2in
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_0.pdf}}
% \caption{Snapshot \# 1 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_0}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_1.pdf}}
% \caption{Snapshot \# 2 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_1}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_2.pdf}}
% \caption{Snapshot \# 3 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_2}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_3.pdf}}
% \caption{Snapshot \# 4 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_3}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_4.pdf}}
% \caption{Snapshot \# 5 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_4}
% \end{center}
% \vskip 0.2in
% \end{figure}

%\twocolumn
\subsection{Embedding Network architectures}
%The BCNN was implemented in TensorFlow (Keras and tensorflow probability) while the SBI package (SNPE-C) makes use of PyTorch. 
Implementations follow the description in Section \ref{sec:arch_ma2}. The architectures are shown in Table \ref{tab:lv}.
\begin{table}[ht]
\caption{Architecture BCNN (top) and SNPE-C(bottom).}
\label{tab:lv}
%\vskip 0.15in
\begin{center}
%\begin{small}
%\begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 51, 2)]&0\\
Conv1DFlipout (ReLU) &(None, 51, 20)  &260\\
MaxPooling1D    &(None, 10, 20)&0\\
Flatten    & (None, 200)&0\\
DenseFlipout (Softmax)     &(None, 25)&10025\\
\midrule
Total weights &&10285

\end{tabular}
%\end{sc}
%\end{small}
%\end{center}

%\caption{Architecture SNPE-C, icluding embedded summary statistics network and neural density estimator (NDE) with 10 hidden units and 2 transforms}
% \label{sample-table}
% \vskip 0.15in
% \begin{center}
% \begin{small}
% \begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 51, 2)]&0\\
Conv1d (ReLU) &(None, 51, 20)  &140\\
MaxPooling1d    &(None, 10, 20)&0\\
view (Flatten)    & (None, 200)&0\\
Linear (ReLU)     &(None, 25)&1608\\
NDE (MAF) &10 hidden, 2 transform&912\\
\midrule
Total weights &&2660

\end{tabular}
%\end{sc}
%\end{small}
\end{center}
%\vskip -0.1in
\end{table}

\subsection{Hyperparameters of BNN}
\label{sec:hyperLV}
As in the case of the MA(2) test problem, the hyperparameters of interest to explore in the context of the BNN aproach for LV include the number of bins (classes) and the extent of exploitation controlled by the threshold $\delta$. In Figure~\ref{fig:lv_bins}, we explore three values for the bins. As the number of bins for the discretization of continuous data increases, so does the convergence speed towards a lower MSE.
%One could theorize that they all will converge to the same MSE, however, at different rates. 

In Figure~\ref{fig:lv_thresh}, we compare the MSE for 8 rounds using a threshold value of $\delta = 0.05$ and $\delta = 0$, i.e., with or without exploitation. It can be seen that incorporating exploitation via $\delta > 0$, results in a more rapid decline in MSE as compared to the case of no exploitation. This is expected, validates the purpose of exploitation in the proposed approach. 
%As in the case of the number of bins, the two cases might converge to the same MSE; however, when using exploitation $\delta > 0$, predictably, one reaches a lower MSE faster than without.

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_bins.pdf}}
\caption{MSE for BCNN with different number of classes corresponding to the square of the number of bins per parameter, on the LV test problem.}
\label{fig:lv_bins}
\end{center}
\end{figure}

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_thresh.pdf}}
\caption{MSE for BCNN when using threshold $\delta$: $0.05$ and $0.0$, on the LV test problem.}
\label{fig:lv_thresh}
\end{center}
\end{figure}

\subsection{Execution time}
Figure~\ref{fig:lv_time} presents the execution time of the BCNN and SNPE-C approaches. However, we note that several factors affect this comparison. It can be observed that the BCNN approach is slower as compared to SNPE-C owing to the overhead of training times for multiple classifiers.
%One should observe the execution time presented in Figure~\ref{fig:lv_time}  with caution. 
The number of required classifiers depends upon the number of parameter pair-combinations, e.g., three parameters requires four models, four parameters require six models, etc. The additional training time for the BCNN in case of LV can be reduced via better parallelization of the training process of the three classifiers. Please refer to Figure~\ref{fig:ma2_time} for a comparison of execution times for a single trained model.

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_time.pdf}}
\caption{Time per round (minutes) and cumulative time (minutes), for BCNN and SNPE-C, on the LV method.}
\label{fig:lv_time}
\end{center}
\end{figure}


\newpage
\onecolumn
%\begin{landscape}
\begin{sidewaysfigure}[ht]
\centering
\includegraphics[width=21cm]{img/LV_grid_seeds.pdf}
\caption{Snapshot \# 1-5 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
\label{fig:lv_grid}
\end{sidewaysfigure}
%\end{landscape}
\end{appendix}

\end{document}

%Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space. Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account.This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions.An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network.This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.


% This document was modified from the file originally made available by
% Pat Langley and Andrea Danyluk for ICML-2K. This version was created
% by Iain Murray in 2018, and modified by Alexandre Bouchard in
% 2019 and 2021. Previous contributors include Dan Roy, Lise Getoor and Tobias
% Scheffer, which was slightly modified from the 2010 version by
% Thorsten Joachims & Johannes Fuernkranz, slightly modified from the
% 2009 version by Kiri Wagstaff and Sam Roweis's 2008 version, which is
% slightly modified from Prasad Tadepalli's 2007 version which is a
% lightly changed version of the previous year's version by Andrew
% Moore, which was in turn edited from those of Kristian Kersting and
% Codrina Lauth. Alex Smola contributed to the algorithmic style files.

NOTINUSEmain.tex
%%%%%%%% ICML 2021 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%%

\documentclass{article}

% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{booktabs} % for professional tables

% Added by authors
\usepackage{amsmath}
\usepackage{amssymb}

% hyperref makes hyperlinks in the resulting PDF.
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
% please comment out the following usepackage line and replace
% \usepackage{icml2021} with \usepackage[nohyperref]{icml2021} above.
\usepackage{hyperref}

% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Use the following line for the initial blind version submitted for review:
%\usepackage{icml2021}

% If accepted, instead use the following line for the camera-ready submission:
\usepackage[accepted]{icml2021}

% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{Robust and integrative Bayesian neural networks for likelihood-free parameter inference}

\begin{document}

\twocolumn[
%\icmltitle{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\icmltitle{Robust and integrative Bayesian neural networks\\ for likelihood-free parameter inference}
% It is OKAY to include author information, even for blind
% submissions: the style file will automatically remove it for you
% unless you've provided the [accepted] option to the icml2021
% package.

% List of affiliations: The first argument should be a (short)
% identifier you will use later to specify author affiliations
% Academic affiliations should list Department, University, City, Region, Country
% Industry affiliations should list Company, City, Region, Country

% You can specify symbols, otherwise they are numbered in order.
% Ideally, you should not use this facility. Affiliations will be numbered
% in order of appearance and this is the preferred way.
\icmlsetsymbol{equal}{*}

\begin{icmlauthorlist}
\icmlauthor{Fredrik Wrede}{equal,uu}
\icmlauthor{Robin Eriksson}{equal,uu}
\icmlauthor{Richard Jiang}{ucsb}
\icmlauthor{Linda Petzold}{ucsb}
\icmlauthor{Stefan Engblom}{uu}
\icmlauthor{Andreas Hellander}{uu}
\icmlauthor{Prashant Singh}{umu}
\end{icmlauthorlist}

\icmlaffiliation{uu}{Department of Information Technology, Uppsala University, Uppsala, Sweden}
\icmlaffiliation{ucsb}{Department of Computer Science, University of California-Santa Barbara, Santa Barbara, CA, USA}
\icmlaffiliation{umu}{Department of Computing Science, Umeå University, Umeå, Sweden}

\icmlcorrespondingauthor{Fredrik Wrede}{fredrik.wrede@it.uu.se}
\icmlcorrespondingauthor{Prashant Singh}{prs@cs.umu.se}

% You may provide any keywords that you
% find helpful for describing your paper; these are used to populate
% the "keywords" metadata in the PDF but will not be shown in the document
\icmlkeywords{Machine Learning, ICML}

\vskip 0.3in
]

% this must go after the closing bracket ] following \twocolumn[ ...

% This command actually creates the footnote in the first column
% listing the affiliations and the copyright notice.
% The command takes one argument, which is text to display at the start of the footnote.
% The \icmlEqualContribution command is standard text for equal contribution.
% Remove it (just {}) if you do not need this facility.

%\printAffiliationsAndNotice{}  % leave blank if no need to mention equal contribution
\printAffiliationsAndNotice{\icmlEqualContribution} % otherwise use the standard text.

\begin{abstract}
State-of-the-art neural network-based methods for learning summary statistics have delivered promising results for simulation-based likelihood-free parameter inference.
Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account. 
This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions. 
An adaptive sampling scheme selects simulation locations to efficiently and iteratively refine the predictive posterior of the network conditioned on observations.
This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.
% Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space.
% % ** 1 **
% Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account. 
% This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions.
% %This work proposes an uncertainty-enhanced summary statistics learning framework using Bayesian neural networks, which incorporates epistemic uncertainties to quantify the confidence of the trained model. 
% % ** 2 **
% An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network.
% %conditioned on observations, hence eliminating the need for discrepancy-based rejection or density estimation and reducing the number of simulations required. 
% % ** 3 **
% %Our novel approach also includes estimating the posterior using categorical distributions, subsequently transforming the learning task into a classification problem. 
% This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.
\end{abstract}

\section{Introduction}
\label{sec:intro}
 Likelihood-free inference (LFI) is a class of methods designed for parameter inference of models with intractable or expensive likelihoods. 
 %The type of models range from ... 
 Approximations to the posterior distribution conditioned on observed data are made with the aid of forward simulations of the model, thus also referred to as simulation-based inference \cite{Cranmer_frontierSBI}. Approximate Bayesian computation (ABC)~\cite{sisson2018handbook} is a popular method in this setting, and involves the use of a discrepancy measure to compare simulated data against observed data. A threshold on the discrepancy measure decides whether a proposed parameter can be accepted as a sample from the posterior. 
 
While this approach has shown to be robust, it has certain drawbacks. For many dynamical models, the simulation output is high-dimensional, 
and one uses summary statistics as a dimension reduction technique
 % avoid large approximation errors
 \cite{wood2010statistical,engblom2020bayesian}. Selecting informative summary statistics so that the discrepancy measure can accurately accept posterior samples is often a tedious task, typically requiring detailed domain knowledge and trial and error. 
 
 An alternative approach to handpicking summary statistics is to utilize embedded neural networks (NN) to learn summary statistics from simulated data \cite{jiang2017learning, dinev2018dynamic, wiqvist2019partially, aakesson2020convolutional,chen2020neural}.  Some of these methods require a post-processing ABC scheme to yield the posterior, others work in conjunction with neural density estimators \cite{lueckmann2021benchmarking} to either estimate the posterior directly, or use synthesized likelihoods which requires Markov chain Monte Carlo (MCMC) sampling. Although deterministic embedded networks for learning summary statistics have shown impressive results, they are prone to be inaccurate in regions where the training data is sparse or not well represented (particularly in the case of problems with high-data dimensionality). 
 %The uncertainty of prediction or network prediction confidence is not taken into account in the LFI pipelines.
 %they can often lead to too `overconfident' predictions\textcolor{red}{cite}. Although implementations of embedding networks include regularization techniques to avoid overfitting, they still suffer from inaccuracies in regions of the parameter space not well-represented in the training data. 
Deterministic NNs do not directly provide an estimate of their prediction uncertainty, which can potentially mislead the parameter inference process.
%Deterministic NNs cannot quantify uncertainties associated with unseen data independently, which manifests as an inherent problem when one samples a finite amount of parameter proposals from large, potentially uninformative priors. 
This lack of direct uncertainty quantification motivates the design of more robust and expressive embedding networks, that can drive the LFI process in a more reliable manner.
 
 % ToDo for PS: elaborate more on the overconfident part - this is due to (limited) sampling in hd spaces, with large gaps of no data
 % PS: Perhaps we should clearly say - Approach 1: (BNN) and Approach 2: (BNN + classification)
This paper presents a novel likelihood-free parameter inference framework that improves upon two characteristics of embedded NN-based LFI methods, as outlined below.
\begin{itemize}
    \item Bayesian embedded NNs are proposed as summary statistics that provide prediction uncertainties, and also directly approximate the posterior. An adaptive sampling algorithm utilizes these uncertainty estimates to iteratively refine the approximated posterior. Including prediction uncertainty in the LFI process provides robustness against potential gaps in training data.
    \item As the Bayesian embedded NN directly models the posterior distribution, no separate post-processing (e.g., density estimation or MCMC) is needed. 
\end{itemize}
 
%This work explores Bayesian embedded networks as trainable summary statistics that provide prediction uncertainties, and directly estimate the posterior distribution.
%Introducing epistemic uncertainties in the embedded network will enable a more robust LFI workflow by considering model confidence in addition to model predictions. The embedded network's predictive posterior will directly estimate the posterior distribution for the inference task, ultimately integrating embedded summary statistics networks with neural density estimators. 
The proposed approach involves approximating the posterior through discrete categorical distributions, subsequently solving a regression problem via classification. This approach in similar form has delivered encouraging results~\cite{weiss1995rule, torgo1996regression, AHMAD_RvC}. The main motivation behind this design is that performing regression with high-dimensional inputs (such as dynamical simulation responses) to continuous multi-dimensional outputs (such as parameters) is a highly challenging task. Even though NNs have the learning capacity to do so, generalizing the networks often proves difficult. 
%Particularly if the training data is not fixed over individual training procedures, such as the case for parameter inference. It's our intention to make parameter inference using NNs more robust by utilizing regression via classification.     

 We demonstrate the efficiency and robustness of our approach on benchmark problems and compare against the deterministic counterparts in conjunction with neural density estimators. Experimental results show  that the proposed approach is consistently more robust than existing methods, with relatively small variance in parameter inference quality performance. The error in parameter estimation is comparable to existing methods. %\textcolor{red}{Think more about how we add value over SNPE}
 %As an alternative approach, some methods have been focusing on introducing inverse mapping functions $f$.  By fitting the inverse mapping function $f(\bold{y}, w) \mapsto \hat{\theta}$ using simulated data  $X = \{y_i, \theta_i\}^N$, a loss function $L(\theta_i, \hat{\theta_i})$ can be optimized w.r.t $w$ between predicted and true parameters, the weights $w$ should ultimately incorporate the discrepancy between samples. While some variants can directly yield density estimators $f(y_0,w) \mapsto \hat{p}(\theta|y_0}$, others use it in a post-processing rejection scheme by letting $d(f(y_i,w), f(y_0,w))$. Learning the inverse mapping function is typically referred as a regression problem where one of the most popular approaches is to use artificial neural networks (ANN).}

%However, most applied deep learning architectures are deterministic and non-probabilistic and can yield overconfident predictions on targets. Further, classification as shown to be superior to regression in prediction performance when $f$ inherits complex non-linear patterns. Thereof, we explore using Bayesian neural networks to better account for epistemic uncertainty and regularization in the mapping function and we discretize $\theta$ space into categorical classes to transform the problem into a classification task, assuming that sample paths $y$ to be similar when in parameter proximity. The inherent properties of dynamical processes makes convolutional neural networks (CNN) an attractive option for learning the mapping function $f$. CNNs make use of a filtering kernel to create local feature maps.Previous studies have demonstrated the capabilities of using determinstic CNNs for likelihood-free parameter inference \cite{}. In this study we use a similar architecture but make use of probablistic layers in Tensflow probability package to place parameterized distributions over the weights of the neural network and apply variational inference using evidence lower bound (ELBO) and \textit{Bayes by Backprop} to fit a posterior distribution. This approach suggests that, not one, but an ensemble of neural networks are learned, where each network's weights are sampled from the fitted posterior. While it might seem computationally heavy learning an infinite ensemble of networks, in fact the amount of weights only increases by a factor of two when using Monto Carlo estimates of the gradients on an otherwise intractable problem.
We organize the paper as follows: Section~\ref{sec:background} introduces the problem setting, notation, and related work. Section~\ref{sec:method} presents the proposed LFI parameter estimation approach. Section~\ref{sec:experiments} demonstrates the proposed methodology on benchmark examples and draws comparisons to other methods. Section~\ref{sec:discussion} presents a discussion on the merits and applicability of the approach. Section~\ref{sec:conclusion} finally gives concluding remarks about the novel method presented in the paper.

\section{Background} %  notation, estimate PDF with Neural net (with embedding net)
\label{sec:background}
% problem set-up | we want the posterior given model M, and observation y0
Let ${\bf y}_0$ be a target observation which can be approximately modeled by $M({\bf y} | {\boldsymbol \theta})$.
%Assume a dynamical model  $\mathcal{M}$ which has generated an observation ${\bf y}_0$. Next, assume some model error by approximating the model by $M({\bf y} | {\boldsymbol \theta})$, with parameters $\boldsymbol \theta$.
Forward simulation of the model $M$ results in the sample path ${\bf y}^{1:T}$ henceforth referred to as ${\bf y}$.
%Let $ {\bf y}_0 \sim M({\bf y}| {\boldsymbol \theta})$ be observed data approximately sampled from $M({\bf y}|\boldsymbol{\theta})$, i.e., we assume some model error.
%The observed data may also have been obtained from the physical process corresponding to $M$. 
The objective is to infer the parameters conditioned on the observation, i.e., the posterior density $p(\boldsymbol{\theta} | {\bf y}_0) \propto p({\bf y}_0 | \boldsymbol{\theta})p(\boldsymbol{\theta})$. However, as we assume that the likelihood is computationally intractable, which is often the case for dynamical models, an estimate $\hat{p}(\boldsymbol{\theta} | {\bf y}_0)$ of the true posterior distribution must be approximated. Approaches in this setting typically utilize access to the observed data  ${\bf y}_0$ and the ability to perform simulations from model $M$ at well-chosen parameter locations. We summarize some the methodologies under the supposed setting below.
%, and motivate the need for a fully Bayesian model-driven LFI approach.


 
 

\subsection{Related work}
% --- Simulation based Inference --- %
% discrepancy-, SS based
The ABC family of methods forms the most popular and established LFI approach (introduced in Section~\ref{sec:intro}). 
%The methodology consists of a sample-simulate-compare cycle where parameter samples $\boldsymbol{\theta}_i \sim p(\boldsymbol{\theta})$ drawn from a defined prior distribution $p(\boldsymbol{\theta})$ are simulated using the model $M$ to obtain sample paths ${\bf y}$, which one consequently compare to observed data ${\bf y}_0$ to either declare T1 as being accepted as part of the estimated posterior or rejected. The workflow consists of several hyperparameters, including the choice of the prior P1, dimension reducing summary statistics (when ${\bf y}$ is high dimensional, e.g., time series), a distance or discrepancy measure to compare simulated and observed data, and a tolerance bound $\epsilon$ to define acceptance/rejection. 
Although remarkably successful, ABC methods are typically very challenging to setup optimally for accurate inference quality and convergence speed \cite{sisson2018handbook,marin2012approximate}. Extensions such as sequential Monte Carlo methods (ABC-SMC) \cite{sisson2007sequential} can potentially accelerate the convergence speed but still require laborious hyperparameter selection. In particular, the choice of informative summary statistics is critical for practical and accurate parameter inference as it forms the means of comparing simulated proposals with the observation. 

Consequently, summary statistic selection has received considerable attention \cite{sisson2018handbook}[Chapter 5]. However, traditional summary statistics such as statistical moments may not always be expressive enough to capture intricate details in rich dynamical data. Further, ideal statistics may even be missing from the candidate pool of statistics to select from. A recent development has been the use of data-driven regression models to extract features from data, and function as summary statistics. It has been shown that the ideal summary statistic is the posterior mean, when minimizing the quadratic loss \cite{fearnhead2012constructing}. In particular, NN based regression models have delivered highly promising results towards learning the estimated posterior mean as summary statistics from high-dimensional detailed time series \cite{jiang2017learning,wiqvist2019partially,aakesson2020convolutional}. Such models learn the mapping $F_{\bf w}({\bf y}) \mapsto \mathbb{E}[\boldsymbol{\theta}]$ which can be seen as an amortized point estimate for a given ${\bf y}$, where ${\bf w}$ are the weights of the regression model. However, an overhead associated with \emph{trainable} summary statistics is the need for large amounts of simulated training data to yield an accurate  posterior mean estimate. Although there are opportunities for re-using the training data in the later rejection sampling step to estimate the conditioned posterior, the sheer amount of training data required is problematic whenever the simulations are expensive to compute. % Mention that they can't quantify uncertainties about their estimates 

%Synthetic likelihood based approaches \cite{wood2010statistical}  step away from discrepancy distances and approximate the likelihood conditioned on the summary statistics.

% remove SS, still discrepancy-dependent
%Efforts to use embedding nets to compute optimal summarizing statistics have shown great promise, NN \cite{jiang2017learning}, Convolutional NN \cite{aakesson2020convolutional}, but they still necessitate a discrepancy distance to evaluate proposals. These approaches consists of fitting a regression model $F_w(y) \mapsto \theta$ parameterized by the weights $w$ and subsequently use prediction on simulated data
%\textcolor{red}{FW: need to introduce SNPE-C in particular\cite{greenberg2019automatic}}
Neural density estimators form a family of methods that directly estimate densities with the aid of NNs and have shown impressive performance on challenging problems \cite{lueckmann2021benchmarking}, such as the masked autoregressive flows (MAF) and the neural spline flow (NSF) \cite{papamakarios2017masked,NSF}. While some neural density estimators are used to estimate posterior densities \cite{APT_2019,SNPE_A_Papamakarios, SNPE_B_lueckmann2017flexible}, others rely on constructing synthesized likelihoods or ratios which require MCMC sampling to extract the posterior \cite{papamakarios2019sequential, lueckmann2019likelihood,SNRE_a, SNRE_b_constrative}. The formed posteriors are amortized, i.e., any observation can be conditioned on the posterior. 

However, a collection of algorithms based on neural density estimators uses sequential Bayesian updates to the posterior, conditioned on a particular target observation. These include sequential neural posterior estimation (SNPE) \cite{SNPE_A_Papamakarios, SNPE_B_lueckmann2017flexible, APT_2019},  sequential neural likelihood estimation (SNLE) \cite{papamakarios2019sequential, lueckmann2019likelihood} and sequential neural ratio estimation (SNRE) \cite{SNRE_a, SNRE_b_constrative}. A benchmarking effort was recently conducted comparing the performance of these approaches on various problems \cite{lueckmann2021benchmarking}. These approaches also support the use of embedding networks to turn high-dimensional simulation outputs to informative summary statistics. The weights of the embedding networks are then trained together with the weights of the neural density estimator. Thus, these shared architectures enable sequential updates on the conditioned posterior and the summary statistics learning process, ultimately reducing the amount of simulations required. 

SNPE-C (also known as automatic posterior transform - APT) \cite{APT_2019} is a recent SNPE technique that learns to infer the true posterior by maximizing an intermediate estimated proposal posterior. SNPE approaches form a good class of methods to compare the proposed approach to, as the use of sequential Bayesian updates to refine the estimated posterior is close in spirit to adaptive sampling used in this work. SNPE approaches have also been shown to perform well across a variety of test problems \cite{lueckmann2021benchmarking,APT_2019}. Among SNPE approaches, SNPE-C is superior as it is more flexible and scalable as compared to SNPE-A and SNPE-B \cite{APT_2019}, and is used for the purpose of comparisons in the experiments in this paper. 


%The most recent and superior version of SNPE is the Automatic Posterior Transform (APT) \cite{APT_2019}, referred as SNPE-C in the software package SBI\footnote{https://www.mackelab.org/sbi/} which is the naming we will be using henceforth.       
    
% A list, that then has to be made into a flowing text.
% *classification*
%Using classification in LFI is not novel~\cite{gutmann2018likelihood}.  
%
% *flow-based*
%Flow-based model $q(\eta|x)$ condition on $x$ to generate the next training data set $\mathcal{D}$ in order to approximate the posterior $p(\eta|x)$, APT (SNPE-C) \cite{greenberg2019automatic, gonccalves2020training}
%Flow-based model with $q(\eta|x)$ conditioned on $\eta$ \cite{papamakarios2019sequential}
%SNPE \cite{lueckmann2017flexible,lueckmann2019likelihood,papamakarios2017masked}
% --- collecting ---
%Benchmarking effort of SBI \cite{lueckmann2021benchmarking}, 

\subsection{Motivation for Bayesian neural networks}
We consider a different approach to neural density estimation. We implement a shared, embedded network for summary statistics, with density estimation through Bayesian neural networks (BNNs). The introduction of weight uncertainties in NNs has enabled a robust means for learning an ensemble of models \cite{Bayesbybackprop}, represented by the posterior density $p({\bf w}|{\bf y})$ where ${\bf w}$ are the network weights.
%Blundell \textit{et al.} demonstrate with their Bayes by backprop algorithm how one can obtain and fit an ensemble of models represented by the posterior density $p(w|X)$ where $w$ is the network weights and $X$ the training data. 
It has also been demonstrated \cite{Bayesbybackprop} that the number of weights only increases by a factor of two by utilizing Monto Carlo (MC) sampling of gradients when updating the weights. The benefit of using a BNN is that the posterior predictive distribution $p(\boldsymbol{\theta}|{\bf y}) = \int p(\boldsymbol{\theta}|{\bf w}, {\bf y})p({\bf w}|{\bf y})d{\bf w}$ provides an uncertainty measure to its predictions. The uncertainty estimates form an elegant means of identifying and countering overfitting, which translates into more robust and efficient embedding networks. Different embedding network architectures such as RNNs or CNNs can be implemented in the BNN setting to provide prediction uncertainty estimates, and directly estimate the posterior $p(\boldsymbol{\theta}|{\bf y}_0) \approx p(\boldsymbol{\theta}|{\bf y})$ by the posterior predictive distribution. The uncertainty estimates also allow sequential adaptive sampling of the posterior, conditioned on a target observation for faster convergence, similar to SNPE. 

%However, due to the integral in the predictive posterior being analytically intractable, numerical integration needs to be utilized (e.g., using MC methods). For the application of LFI, a BNN will provide the flexibility of using embedding networks, e.g., architectures such as RNNs or CNNs, that provide uncertainty estimates of predictions, and directly estimate the posterior $p(\theta|y_0) \approx p(\theta|X)$ by the posterior predictive distribution. This, in turn will enable sequential adaptive sampling of the posterior conditioned on a target observation for faster convergence. 
%\textcolor{red}{FW: Not completely sure of the notation of posterior predictive distribution in our case is it $p(\theta|X)$, $p(y_0|X)$, $p(\theta|w,X)$ etc??} 
%\textcolor{blue}{PS: $p(\theta|X)$, or $p(\theta|w,X)$ (there are arguments on both sides whether or not to include $w$. I'd go with $p(\theta|X)$. It is okay to abstract model specifics (in the form of $w$).}
%The need for large quantities of (potentially computationally expensive) simulated data for training summary statistics and subsequently estimating the posterior distribution using the ABC family of methods can lead to impractical parameter inference processing times. Therefore, this paper aims at developing a data-efficient LFI parameter estimation pipeline that minimizes the training data needed to obtain accurate high-quality summary statistics using an adaptive sampling algorithm, and directly estimates the posterior using Bayesian neural networks thereby eliminating the need for ABC-based posterior estimation. Further, an efficient classification-based approach is also presented that iteratively shrinks the prior search space.
% 
% 

\section{Likelihood-free inference via Bayesian neural networks}
\label{sec:method}
 Let $D = \{({\bf y}_i,\boldsymbol{\theta}_i)\}_1^N$ consist of simulated data and parameter proposals sampled from the prior distribution over the parameter space $p(\boldsymbol{\theta})$. Consider a BNN, parameterized by the weights ${\bf w}$, that computes the posterior density $p({\bf w}|{\bf y})$ of a probabilistic model $p(\boldsymbol{\theta}|{\bf y},{\bf w})$ by placing a prior $p({\bf w})$ on the weights. The posterior predictive distribution for new unseen sample path ${\bf y}_t$,  $p(\hat{\boldsymbol{\theta}}|{\bf y}_t) = \mathbb{E}_{p({\bf w}|D)}[p(\hat{\boldsymbol{\theta}}|{\bf y}_t, {\bf w})]$ is the expectation on unseen data. Setting ${\bf y}_t = {\bf y}_0$, for $\hat{\boldsymbol{\theta}} \approx \boldsymbol{\theta}$ the posterior predictive distribution will be equal to $p(\boldsymbol{\theta}|{\bf y}_0)$ as $N \rightarrow \infty$, which is the target parameter posterior conditioned on a particular observation. Additionally, when the probabilistic model $p(\boldsymbol{\theta}|{\bf y},{\bf w})$ is a regression model, it also will yield $\hat{\boldsymbol{\theta}} = \boldsymbol{\theta}$. However, as we demonstrate next, we will approximate the parameters $\boldsymbol{\theta}$ through discrete categories making $p(\boldsymbol{\theta}|{\bf y},{\bf w})$ a categorical distribution thereby transforming the regression problem into a classification problem. 

\subsection{Probabilistic regression via classification}
Let $p(\boldsymbol{\theta}'|{\bf y},{\bf w})$ be represented as a categorical distribution using cross-entropy and softmax activation in a NN classifier denoted by $F_{p(\boldsymbol{\theta}'|{\bf y},{\bf w})}$.  We will transform the continuous problem by discretizing $\boldsymbol{\theta} \in \mathbb{R}$ into $\boldsymbol{\theta}'\in K$ by $B$ equal width bins per dimension (please see Section~\ref{sec:discussion} for a discussion on binning strategies), i.e., the predictive distribution will provide the probability of belonging to a particular bin $K \in {1,2,...,B}$. By placing a multivariate Gaussian on $p({\bf w})$ we will fit $p({\bf w}|D)$ using Bayes by backprop, subsequently yielding the BNN classifier  $F_{p({\bf w}|D)}$. To estimate the posterior predictive distribution conditioned on our target observation, we sample from $F_{p({\bf w}|D)}$ using a MC sampling scheme,   
\begin{equation}
    p(\boldsymbol{\theta}'|{\bf y}_0) \approx \frac{1}{N}\sum_{i=1}^N F_{p({\bf w}|D)}({\bf y}_0, {\bf w}_i).
\end{equation}

To enable sequential adaptive sampling during the inference task, we will train $G$ subsequent classifiers $F^G_{p({\bf w}|D)}$ where the bins are updated by sampling from the posterior predictive distribution. Each iteration employs the density from the previous generation as the prior for sampling new training data. The BNN generates amortized posterior estimates. However, when conditioned on the observation, it allows for adaptive refinement in each iteration. To be able to simulate new samples, we need to transform $\boldsymbol{\theta}'$ back to $\boldsymbol{\theta}$. This is achieved by transforming each bin $K$ into a uniform distribution,

\begin{equation}
\begin{aligned}
    \{\boldsymbol{\theta}'\}_{i=1}^N &\sim p(\boldsymbol{\theta}'|{\bf y}_0) \\
    \{\boldsymbol{\theta}\}_{i=1}^{N} &= \{\mathbb{U}^K({\boldsymbol{\theta}'}_i^{\text{\emph{left}}},{\boldsymbol{\theta}'}_i^{\text{\emph{right}}})\}_{i=1}^N,
\end{aligned}
\end{equation}

where \emph{left} and \emph{right} are the boundaries of the bin intervals. The adaptive sampling-driven BNN is depicted in Figure~\ref{fig:pipeline} and detailed in Algorithm~\ref{alg:posterior}.

\begin{figure}[t]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/pipeline_v3.pdf}}
\caption{Adaptive sampling-driven BNNs. (A) Parameter proposals are sampled from a prior and simulated via the model. (B) Regression via classification. Continuous proposals are discretized into a categorical distribution with classes being the bins. A BNN (having prior weights) is then trained on data. (C) The posterior predictive conditioned on observation $y_0$ can be estimated via MC sampling, and is subsequently used in an iterative procedure, replacing the previous prior. Note that a transformation is applied to the categorical distribution to transform proposals back to continuous.}
\label{fig:pipeline}
\end{center}
\vskip 0.2in
\end{figure}

\subsection{Exploitation of bins}
%With large priors $p(\boldsymbol{\theta})$, finding the subset $p(\boldsymbol{\theta}|{\bf y}_0)$ can be a slow process.% 
We aim to exploit the uncertainty estimates provided by the posterior predictive distribution to enable faster convergence. A key reason why the process can be slow is that low probability weights linger between iteration updates. We exploit the posterior predictive distribution by using a threshold value, $\delta$. Given the categorical distribution, we set the classes with probability weight below the threshold to zero. The non-zero probabilities are then normalized, constructing a less skewed proposal distribution. Between iterations, $\delta_i$ can be chosen adaptively or kept constant throughout. We recognize that the sequence of distinct $\delta_i$ holds a similarity to the discrepancy thresholds ($\epsilon_i$) used by ABC-SMC~\cite{sisson2007sequential, del2012adaptive}.
% ** if not adaptive **
The potential adaptivity is briefly discussed in Section~\ref{sec:discussion}.
%% *** if we decide on using adaptive ***
%To slow the exploitation, we employ an $\epsilon$ that converges to zero as the rounds increase.



\subsection{Dealing with the curse of dimensionality}
For the single-parameter problem, the structure of the classification problem is straightforward: one classifier for one parameter. However, for multiple parameter dimensionality, we construct one classifier for each two-dimensional covariance combination: $(\boldsymbol{\theta}_i, \boldsymbol{\theta}_j)$, implying that for a model of $k$ parameters, we would require $\frac{k!}{(k-2)!2!}$ classifiers. One sees that the number of classifiers quickly increases; however, they lend themselves towards an embarrassingly parallel workflow since they are independent. Further, the adaptation of the proposals is done per covariance dimension and then marginalized, resulting in the $k$-dimensional parameter proposals.


%The LFI problem is solved by finding the inverse map $F(y)\mapsto \theta$. Consider the approximation $F_w(y)$ produced by an embedded net with weights, $w$. By fitting the inverse mapping function $F_w(y)$ using simulated data $\mathcal{D} = \{(y_i,\theta_i)\}_1^N$, a loss function $\mathcal{L}(\theta_i, \hat{\theta_i})$ can be optimized w.r.t. $w$ between predicted and true parameters; the weights $w$ should ultimately incorporate the discrepancy between samples.
%Our probabilistic approach allows incorporating epistemic uncertainty by employing Bayesian embedding nets (BNN) \cite{mackay1992practical}, which learn distributions of $q(w|\mathcal{D})$ instead of a point estimate $\hat{\theta}$ by introducing a prior $p(w)$ on the weights. Approximating the predictive posterior $p(y_0 |\mathcal{D})$ by Monte Carlo integration gives an interpretation of $\hat{p}(\theta | y_0)$, namely approximating the integral $ p(y_0 |\mathcal{D}))  = \int p(y_0 |\mathcal{D}),w)q(w|\mathcal{D}))dw$
%by a sum of independent samples $ p(y_0 |\mathcal{D})) \approx \frac{1}{M}\sum_{i=1}^M p(y_0 | w_i)$
%where $w_i \sim q(w | \mathcal{D}))$. 
%A connection between the NN and the BNN is that NN's weights can be seen as maximum likelihood estimates or maximum a posteriori estimates, when using regularization. %\cite{}? could use, https://arxiv.org/pdf/2007.06823.pdf but not sure how close they are to be published results.

%\begin{equation}
%    \begin{aligned}
%        p(y_0 |\mathcal{D}))  = \int p(y_0 |\mathcal{D}),w)q(w|\mathcal{D}))dw\\
%         \approx \frac{1}{M}\sum_{i=1}^M p(y_0 | w_i)\\
%         \approx \hat{p}(\theta|y_0)\\
%         \text{where } w_i \sim q(w | \mathcal{D}))
%    \end{aligned}
%\end{equation}


% The classification problem
%\subsection{Accelerating posterior estimation using classification}
%To achieve robust results (\textcolor{red}{FW: this is not enough motivation for using classification}), around the sought after posterior distribution $\hat{p}(\theta|y_0)$, we consider a sequence of discrete mappings from $y \mapsto \theta$ with increasing resolutions. Therefore, we solve finding the continuous map $F$ around $y_0$ by a series of approximate mappings, i.e., $F = f_1 \circ f_2 \circ \dots \circ f_G$. 
%The discrete map $f_k$ considers binned continuous data, meaning, finding $f_k$ is a classification problem for a categorical distribution. We condition each iterative map on $y_0$ to determine the area of interest in the prior, i.e., $f_{k+1}$ follows $f_k$, and $P_{f_k}(\theta|Y_0)$ gives the samples of interest. 
% The final inference step
%A key feature inherent of BNN is that the map $F$ direcred}tly gives the probability $p(\theta|y)$, and given the observation $y_0$, we get $\hat{p}(\theta|y_0)$ without any further steps.


%\section{Classification for LFI using Bayesian neural networks}
%Our approach's contribution is that the BNN allows for uncertainty estimates directly while on the classification step, instead of using the accuracy as a pseudo-measure for the discrepancy between the observation and simulation. We construct the classes by grouping continuous-data into a fixed number of bins. As the number of bins approach the number of samples, we approach the continuous, regression case.  

%The posterior density estimation by BNN is generated iteratively by a sequence of generation densities.  Each iteration employs the previous generation density as it's prior for sampling new training data. The BNN generates amortized posterior estimates. However, when conditioned on the observation, it allows for adaptive refinement in each iteration.

%Denote the trained BNN as $F(y|w)$ with weights $w$, the posterior distribution, approximated by the BNN, $q_{F(y,w)}$. We generate $q_F$ by Monte Carlo-sampling the network, and retrieve a categorical distribution over the binned data.


\begin{algorithm}[]
   \caption{Iterative BNN posterior estimation }
   \label{alg:posterior}
\begin{algorithmic}
   \STATE {\bfseries Input:} model simulator $M({\bf y}|\boldsymbol{\theta})$, 
   observations ${\bf y}_0$, Bayesian neural network $F_{p({\bf w}|D)}$, 
   number of generations $G$, number of simulations per generation $N$,
   prior distribution $p(\boldsymbol{\theta})$, number of bins $B$.

   \STATE $p_0 := p$  
   \FOR{$g=1$ {\bfseries to} $G$}
        %\FOR{$n=1$ {\bfseries to} $N$} % training
            \STATE Sample $\boldsymbol{\theta}_N = \{\boldsymbol{\theta}_i\}_{i=1}^N$ from $p_{g-1}(\boldsymbol{\theta})$
            \STATE Sample ${\bf y}_N = \{{\bf y}_i\}_{i=1}^N$ from $M({\bf y}|\boldsymbol{\theta}_i)$
       % \ENDFOR
        \STATE Group $\boldsymbol{\theta}_N$ into K bins $\boldsymbol{\theta}'_N, \, K= 1:B$
        \STATE Train $F^g_{p({\bf w}|D_g)}$ on $D_g = \{{\bf y}_N, \boldsymbol{\theta}'_N\}$
        \STATE Estimate $p_{g}(\boldsymbol{\theta}'_N|{\bf y}_0) \leftarrow \mathbb{E}[F^g_{p({\bf w}|D_g)}({\bf y}_0,{\bf w}_N)]$
        \STATE Transform $p_g(\boldsymbol{\theta}) \leftarrow \mathbb{U}^K({p}_{g}(\boldsymbol{\theta}'_N|{\bf y}_0))$
        %\STATE \textcolor{red}{FW: Not sure what notation to use for the transform}
    \ENDFOR
    \STATE {\bfseries return} $p_g(\boldsymbol{\theta})$
\end{algorithmic}
\end{algorithm}

%\subsection{Complexity}
%\textcolor{red}{FW: Can we say something more about the complexity of the approach more than the number of weights? E.g simulations $>>$ training time? Also curse of dim, several classifiers are embarrassingly parallel and the number of simulations does not increase. Very important that the reader understand this.} 

%\subsection{Design of the BNN}
%\textcolor{red}{FW: I think this part should go into experiments as a particular case of the BNN tailored for dynamic models}
%In this work we have implemented the BNN as Convolutional NN, henceforth called BCNN. However, we would to point out that many other types of architectures can be used. Out motivation behind this choice is that CNNs have shown great performace in deducting inormation from time-series data \cite{sadouk2019cnn, aakesson2020convolutional}, and not only for images. To turn the layers of a CNN into probabilitic layes we have used the flipout \cite{wen2018flipout} layes supported in Tensorflow \cite{tensorflow2015-whitepaper} probability package. 

%\subsection{Adaptive sampling}


\section{Experiments}
\label{sec:experiments}
%\textcolor{red}{FW: missing transition into CNN (BCNN), have not decided if it should be introduced in section \ref{sec:method} or as a particular case of a BNN in section \ref{sec:experiments}}

To test our approach we have implemented a convolutional BNN (we denoted this particular implementation BCNN). However, we note that our approach is not restricted to these types of architectures.  Our motivation behind choosing a convolutional architecture is that they have demonstrated excellent performance in identifying intricate patterns in complex, high-dimensional time-series data \cite{dinev2018dynamic, sadouk2019cnn, aakesson2020convolutional} as well as images~\cite{APT_2019}. For further information on the implementation we refer the reader to supplementary materials.  


We explore the performance of the BCNN on two benchmark examples - the moving averages model (MA) and the discrete-state continuous-time Markov chain implementation of the Lotka-Volterra (LV) \cite{lotka}. Together with the resulting posterior we recover using the BCNN, we also compare the inference performance to ABC-SMC and SNPE-C with the MAF as the neural density estimator.

The SBI package \cite{tejero2020sbi} implements SNPE-C, and is used for the experiments in this paper. To keep the comparison fair between SNPE-C and the BCNN, we designed the embedded nets to be as similar as possible between the two approaches, resulting in having approximately the same number of weights in the two models.  
 
All of the experiments were conducted on a machine consisting of a 12-core Intel Xeon E5-1650 v3 processor operating at 3.5 GHz, 64 GB RAM, running Fedora core 32 Linux operating system. Even though GPU-acceleration would speed up NN training substantially, in the considered setting we assume that simulations are significantly more expensive than training time. This is the case for the discrete and stochastic LV model, less so for MA.   


\subsection{Moving average model}%- Noisy, high data dimensionality
% \begin{equation}
%     X_t = \epsilon_t + \theta_1\epsilon_{t-1} + \theta_2\epsilon_{t-2},
%     \epsilon_t \sim N(0,1) 
% \end{equation}
The moving average (MA) model is a popular benchmark problem in the ABC setting \cite{marin2012approximate}. The MA model of order $q$, MA($q$) is defined for observations $X_1, \dots, X_p$ as \cite{wiqvist2019partially,aakesson2020convolutional},
\begin{equation}
    X_j = Z_j + \theta_1 Z_{j-1} + \theta_2 Z_{j-2} + \dots + \theta_q Z_{j-q},
\end{equation}
where $j=1,\dots,p$, and $Z_j$ represents latent white noise. For the experiments in this paper, we consider $q=2$ and $Z_j \sim N(0,1)$ \cite{wiqvist2019partially}. The MA(2) model is identifiable in the following triangular region,
\begin{equation}
    \theta_1 \in [-2,2], \theta_2 \in [-1, 1], \theta_2 \pm \theta_1 \geq -1.
\end{equation}
In order to ease implementation and compatibility between the compared approaches, a rectangular uniform prior encompassing the triangular region was used in the experiments. We note here that the estimated posterior was always within the triangular region, therefore no identifiability concerns were encountered due to a rectangular prior.

%\textcolor{red}{TODO: However we use a uniform prior...how do we motivate this choice? Both SNPE-C and ABC-SMC need a pdf of the prior. BCNN don't. Further the SBI package don't have an API supporting user-defined priors, which is baaaad}

\subsubsection{Experimental Setup}
In the experiment we conduct a total of $G=6$ iterations (rounds) of refining the posterior estimate for 10 individual seeds. The total amount of parameter proposals in each round is set to $N=3000$ and the number of bins for BCNN is set to $B=4$ per parameter dimension, resulting in a total of 16 bins. The exploitation threshold is set to $\delta = 0.05$. We simulate the MA(2) model with 100 equidistant time points.
For the ABC-SMC method, we use auto-correlation coefficients of lag 1 and 2, and variance as summary statistics. They are non-sufficient, but are chosen as a representation of a typical choice made by a practitioner in absence of much prior knowledge. Euclidean distance was used as the discrepancy measure.

\subsubsection{Results}
Here we demonstrate the perfomance of the proposed BCNN approach and compare against SNPE-C and ABC-SMC. The model has a superficial posterior, but the map $\boldsymbol{\theta} \mapsto {\bf y}$ is ill-discriminated, i.e., considerable variation in $\boldsymbol \theta$ provides little feedback in ${\bf y}$.  %Figure~\ref{fig:grid_MA2} A shows a qualitative evaluation of two sample executions(seed) of BCNN and SNPE-C. We observed that the BCNN is more robust in estimating the posterior as compared to SNPE-C, which we further demonstrate in Figure~\ref{fig:emd_MA2} by calculating the Earth Moving Distance (EMD, also referred as Wasserstein distance) and Maximum Mean Discrapancy (MMD) between estimated posteriors and the exact posterior. Finally, we also include ABC-SMC for reference; however, this method requires a substantial increase of simulations per round for convergence.  

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_grid_4_25x4_25_fontsize8.pdf}}
\caption{MA(2) per round posterior estimation for BCNN, SNPE-C, and ABC-SMC, together with the ground truth - a MCMC generated posterior. Each round includes $N=3000$ parameter proposals, except for ABC-SMC which require many more. BCNN and SNPE-C includes two different random seeds (1) and (2). Final column to the right shows a Gaussian kernel density estimate of the final round posterior per method.}
\label{fig:grid_MA2}
\end{center}
\vskip 0.2in
\end{figure}

Figure~\ref{fig:grid_MA2} provides qualitative insights on the shapes of the estimated posterior distributions using the different approaches. Figure~\ref{fig:emd_MA2} depicts the log mean earth mover distance (EMD)~\cite{levina2001earth} and the mean of maximum mean discrepancy (MMD) \cite{Scholkopf_MMD} between the estimated posteriors and the exact posterior. We see similar convergence patterns between BCNN and SNPE-C for both the EMD and MMD over subsequent rounds and different seeds, with the BCNN having a slight edge. It can also be observed that the BCNN is highly robust over different seeds with very little variance in performance.
Another interesting observation from Figure~\ref{fig:grid_MA2} is the improvement or refinement extracted by each approach over subsequent rounds. It can be seen that BCNN(1) and BCNN(2) are able to substantially improve the quality of the estimated posterior between rounds 2-4 and 4-6. The ability of the BCNN architecture to greatly improve over successive rounds can be attributed to the combination of uncertainty estimates from the BCNN model leveraged by the adaptive sampling algorithm to identify regions most likely to contain the true posterior.

%The lack of variability between seeds, the error bars in the figure showcase the robust properties of the BCNN method.

\begin{figure}[ht]

\includegraphics[width=\columnwidth]{img/ma2_emd_mmd_4.25x2_fontscale08.pdf}
\caption{Posterior error estimates, log mean EMD (left) and Mean MMD (right) for BCNN, SNPE-C and ABC-SMC per round, with variability $\pm$ 1 std, recovered from ten independent seeds. The BCNN have a slight edge over SNPE-C, especially when it comes to the variability. Note that on average ABC-SMC will require more simulations to converge than the other two, see bottom row in Figure~\ref{fig:grid_MA2}.}
\label{fig:emd_MA2}

\end{figure}

\subsection{Lotka-Volterra predator-prey model}
The Lotka-Volterra predator-prey dynamics model is a popular parameter estimation benchmark problem \cite{aakesson2020convolutional, greenberg2019automatic}. The model is implemented as a stochastic Markov jump process \cite{prangle2017adapting} simulated using the stochastic simulation algorithm (SSA) \cite{gillespie1977exact}. The model consists of three events - prey reproduction, predation and predator death. The model is described by the events:% $\mathcal{X}_1 \longrightarrow 2\mathcal{X}_1$, $ \mathcal{X}_1 + \mathcal{X}_2 \longrightarrow 2\mathcal{X}_2$ and $ \mathcal{X}_2 \longrightarrow \phi$.
\begin{equation}
 \begin{aligned}
     \mathcal{X}_1 &\xrightarrow{\theta_1} 2\mathcal{X}_1,\\
     \mathcal{X}_1 + \mathcal{X}_2 &\xrightarrow{\theta_2} 2\mathcal{X}_2,\\
     \mathcal{X}_2 &\xrightarrow{\theta_3} \emptyset.
 \end{aligned}
\end{equation}
The parameters ${\boldsymbol{\theta}} = \{ \theta_1, \theta_2, \theta_3\}$ control the events described above, and have corresponding rates $\theta_1 \mathcal{X}_1, \theta_2 \mathcal{X}_1 \mathcal{X}_2, \theta_3 \mathcal{X}_2$ respectively. 
\subsubsection{Experimental Setup}
% K = 5
% G = 8
% N = 1000
% \delta = 0.05
We study the inference using a total of $G=8$ rounds of refining the posterior estimate, with five repetitions using different random number seeds, setting the total amount of parameter proposals in each round to  $N=1000$. Further, we define the number of bins to $B=5$ per dimension and the threshold to be $\delta = 0.05$.
The LV model data consists of 51 observations in the time span $[0 \text{ h},50 \text{ h}]$. We construct the model and the corresponding simulator using \textit{GillesPy2}\footnote{https://github.com/StochSS/GillesPy2} \cite{gillespy_2017}. 
Finally, the initial conditions include $\mathcal{X}_1=50$ and $\mathcal{X}_2=100$, and the actual parameters to be inferred are $[log(1.0), log(0.005), log(1.0)]$. The prior we apply per dimension is $\mathbb{U}(log(0.002), log(2))$. 
For the ABC-SMC method, we use no summary statistics, i.e., the data itself is directly compared, and a Euclidean distance as the discrepancy measure.

\subsubsection{Results}
% MSE
Compared to the MA(2) model, we do not have an analytical posterior to compare to, and we cannot refer to the EMD of the method. However, we solve the problem with known truth (the parameters to infer) and consider the mean square error (MSE) for computed posteriors. The reasoning is that MSE holds information both about the bias and the variance of the distribution. In Figure~\ref{fig:MSE_LV}, we evaluate the said MSE of the BCNN approach together with SNPE-C and ABC-SMC. The BCNN performs on par with SNPE-C while seemingly holding to the posterior it finds; see round 8 and the spike in the variance of SNPE-C. 
Exploring what might have caused the increase in variance, we refer to Figure~\ref{fig:grid_LV}, which presents snapshots of a sequence of rounds from one run with the different methods. SNPE-C encounters an instability after round 4. From the MSE, we note that SNPE-C is mostly, not always, stable compared to the empirically robust BCNN.
Finally, we recognize that the initially poor performance of the ABC-SMC method can be attributed to the lack of adequately tuned summary statistics.

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_mse}}
\caption{The lines with error bars show the mean MSE with variability $\pm1$ std, recovered from five seeds with eight rounds for SNPE-C, BCNN, and ABC-SMC. All methods converge to similar MSE at the final round, with low variability. Notable is the initial increase in MSE for ABC-SMC; from Figure~\ref{fig:grid_LV} it can be inferred that the approximated posterior is bi-modal before finding a suitable discrepancy threshold.}
\label{fig:MSE_LV}
\end{center}
\vskip 0.2in
\end{figure}

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_dens}}
\caption{Snapshots of the posterior estimates for the LV problem using BCNN, SNPE-C, and ABC-SMC. Each column is a different round: 2, 4, and the final 8, and each row gives the three parameters combinations possible of $\theta_1$, $\theta_2$, and $\theta_3$. The true parameter value is indicated by the orange cross. The prior covers the entire box uniformly.}
\label{fig:grid_LV}
\end{center}
\vskip 0.2in
\end{figure}


%\subsection{Example 3?}
% placeholder
\section{Discussion}
\label{sec:discussion}
% Gotchas and pitfalls
%\textcolor{red}{PS: Can we mention some pitfalls, something to be careful about and pointers on how to get the best out of our approach?}
% - by using classification we cannot re-use the trained network, we have to start anew each round. Meaning that the information gained from each trained network must be leveraged to it's fullest when constructing the proposal distribution.
% - number of parameter dimensions

The results presented in the previous section demonstrate that the proposed approach is comparable to SNPE-C, which is a state-of-the-art method for approximating flexible posterior densities using NNs. The uncertainty estimates and a simplification strategy of accomplishing regression via classification using the BNN allows the adaptive sampling algorithm to consistently drive the search for the true posterior in the right direction. This can be seen in the very low standard deviations presented in Figures~\ref{fig:emd_MA2} and \ref{fig:MSE_LV}. 
However, an important observation is that SNPE-C does not utilize any exploitation similar to $\delta$ in the proposed approach  (see Section~\ref{sec:method}). Effectively, the deviations in performance are reduced using this exploitation. 

However, incorporating a similar exploitation strategy used in SNPE methods may not alone resolve the issue of occasional, substantially inaccurate posterior estimates, such as SNPE-C(2) in Figure \ref{fig:grid_MA2}. The stability and robustness of the proposed approach stems from the Bayesian setting considered in this work. Further, the BNN is specifically tailored for models with complex data distributions, such as discrete stochastic and dynamical models. The BNN functions as both  an embedding network to extract summary statistics and as a posterior estimator. However, the use of categorical distributions as posterior estimators is limited to fairly simplistic posteriors. For flexible posterior estimators we recommend SNPE-C, but to utilize the robustness and the exploitative properties of the proposed approach one might consider using both approaches in conjunction with each other. For example, using the BNN as a pre-processing step to robustly define a prior distribution, and subsequently passing the posterior estimate $p(\boldsymbol{\theta}'|y_0)$ as the prior for SNPE-C. This approach is outlined in Algorithm~\ref{alg:proposed}.

\begin{algorithm}[]
   \caption{Proposed: BNN in conjuction with SNPE-C}
   \label{alg:proposed}
\begin{algorithmic}
   \STATE Do Algorithm~\ref{alg:posterior} to get $\tilde{p}_G(\boldsymbol{\theta})$
   \STATE Do SNPE-C with $\tilde{p}_G(\boldsymbol{\theta})$ as prior
\end{algorithmic}
\end{algorithm}




% Significance of uncertainty estimates provided by the B(C)NN
%The results presented in the previous section underscore the importance of accounting for embedding model uncertainty towards the robustness of resulting LFI quality. The uncertainty estimates provided by the BNN allow the adaptive sampling algorithm to consistently drive the search for the true posterior in the right direction. This can be seen in the very low standard deviation of parameter inference error shown in Figures~\ref{fig:emd_MA2} and \ref{fig:MSE_LV}.

% Significance of adaptive sampling and categorical classification
%Further, adaptive sampling also enables consistent and stable refinement of the estimated posterior over successive rounds of algorithm execution. \textcolor{red}{PS: @Fredrik: pls put in some text here connecting how adaptive sampling and categorical classification is responsible for the refinement. Further, would be good to contrast with SNPE and talk about our advantages}

% Dynamical models
%\textcolor{red}{PS: why is our approach well-suited for dynamical models}
% + find "optimal" summaries, which is a hard task for time-series/dynamical models. Able to handle high dimension data well. 
\subsection{Future work}
% Future work
%\textcolor{red}{PS: Future work, explore more sophisticated sampling algorithms and the interplay between exploration and exploitation}

The convergence plots in Figure~\ref{fig:emd_MA2} and \ref{fig:MSE_LV} point towards an opportunity  for early stopping when the posterior estimate no longer receives significant updates. A possibility could be to measure the highest density region (HDR)~\cite{HDR}, or EMD, change between posterior estimates, and stop if the difference is below a threshold value. Using this could substantially reduce the number of simulations involved and remove the requirement of specifying the amount of rounds $G$.

Optimizing the number of bins $B$ as well as the number of proposals $N$ in each bin (support) is currently not trivial. A sufficient support for dynamical models is highly dependent on the response of simulation outputs to perturbations in the parameter space. For the MA(2) model, a larger $N$ per round was required, as well as fewer bins compared to the settings for the LV problem. This was owing to the MA(2) model mapping being ill-discriminated in nature as described earlier. Thus, for the BNN classifier to sufficiently learn discriminating features, one would require more data. Further, currently the bin width is fixed and does not consider the distribution of proposals. Exploring variable bin widths based on the rate of change of proposal distributions over successive rounds could potentially improve convergence speed, and also balance the support for bins more evenly. 

The procedure of assigning a continuous $\theta_i$ into a bin does not take discretization uncertainty into account (e.g., when $\theta$ is close to the egde of two bins), which can lead to inaccurate training. The aforementioned problems will be considered as future work and will involve adaptive bin selection based on the discriminative ability of the BNN. Further, bin uncertainties could potentially be tackled using fuzzy set theory and other bin strategies used in regression via classification.

The hyperparameter $\delta$ (see Section~\ref{sec:method}) that controls exploitation is currently considered to be fixed. However, an adaptive $\delta$ dependent on the current round can potentially improve convergence, e.g., an exponentially decaying $\delta$. The similarity to the threshold one use in ABC-SMC, aligns the adaptivity of $\delta$ well with strategies already in practice.

 %\textcolor{red}{FW: please elaborte, I don't understand}
 %One could potentially consider making it an adaptive procedure, dependent on how well the BNN resolves the previous step; if one observes a low effective sample size, then decrease the number of bins and vice versa. Additionally, currently, the bins are of equal width per dimension; by leveraging information from the proposal distribution, one could explore more informative bin widths, potentially bridging learning between rounds.


%discussion about early stopping, constructing bins, number bins and suggested exploitation. (the number of weights in BNN increase as number of bins increase, fuzzy bins?



\section{Conclusion}
\label{sec:conclusion}
This paper presented a novel likelihood-free parameter inference framework for parameter inference of dynamical simulation models. The proposed approach leverages Bayesian neural networks for learning high-quality summary statistics as the estimated posterior while also incorporating epistemic uncertainties. This allows for a posterior distribution estimate without the need for a post-processing step such as rejection sampling. An adaptive sampling algorithm utilizes the estimated posterior and uncertainty estimates from the Bayesian neural network to iteratively shrink the prior search space for the actual posterior distribution. An efficient classification-based density estimation approach further approximates the posterior. The proposed approach is compared to the current state-of-the-art sequential neural posterior estimation method and the well-established approximation Bayesian computation sequential Monte Carlo likelihood-free inference methods. Experiments performed on benchmark test problems show that the proposed approach is consistently more robust under various situations compared to existing methods while delivering comparable parameter inference quality.

\bibliography{references}
\bibliographystyle{icml2021}

\end{document}

%Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space. Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account.This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions.An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network.This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.


% This document was modified from the file originally made available by
% Pat Langley and Andrea Danyluk for ICML-2K. This version was created
% by Iain Murray in 2018, and modified by Alexandre Bouchard in
% 2019 and 2021. Previous contributors include Dan Roy, Lise Getoor and Tobias
% Scheffer, which was slightly modified from the 2010 version by
% Thorsten Joachims & Johannes Fuernkranz, slightly modified from the
% 2009 version by Kiri Wagstaff and Sam Roweis's 2008 version, which is
% slightly modified from Prasad Tadepalli's 2007 version which is a
% lightly changed version of the previous year's version by Andrew
% Moore, which was in turn edited from those of Kristian Kersting and
% Codrina Lauth. Alex Smola contributed to the algorithmic style files.

abstract.tex
\documentclass[a4paper]{article}
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{float}
\usepackage[font=small,labelfont=bf]{caption}
\usepackage[a4paper,top=1cm,bottom=2cm,left=4cm,right=4cm,marginparwidth=1.75cm]{geometry}
\begin{document}
%\title{Making Sense of Large Computational Experiments}
\title{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\date{}
%Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference
%Likelihood-free parameter inference using supervised sequential bayesian neural networks
%\author{F. Wrede, R. Eriksson, R. Jiang, L. Petzold, A. Hellander, P. Singh}
%\footnote{Presenting author.}
\maketitle
\baselineskip=0.9\normalbaselineskip
\vspace{-10pt}

State-of-the-art neural network-based methods for learning summary statistics have delivered promising results for simulation-based likelihood-free parameter inference. This work proposes an uncertainty-enhanced summary statistics learning framework using Bayesian neural networks, incorporating epistemic uncertainties to quantify the confidence of trained models. An adaptive sampling scheme selects simulation locations to efficiently and iteratively refine the predictive posterior of the network conditioned on observations, eliminating the need for discrepancy-based rejection and reducing the number of simulations required. Our novel approach includes estimating the posterior using categorical distributions, subsequently transforming the learning task into a classification problem, thus allowing for efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against similar state-of-the-art methods. 

\end{document}
suppl.tex
\documentclass{article}

% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{rotating}
\usepackage{booktabs} % for professional tables

% Added by authors
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{pdflscape}

% hyperref makes hyperlinks in the resulting PDF.
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
% please comment out the following usepackage line and replace
% \usepackage{icml2021} with \usepackage[nohyperref]{icml2021} above.
\usepackage{hyperref}

% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Use the following line for the initial blind version submitted for review:
\usepackage{icml2021}

% If accepted, instead use the following line for the camera-ready submission:
%\usepackage[accepted]{icml2021}

% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{Robust and integrative Bayesian neural networks for likelihood-free parameter inference
}

\begin{document}

\onecolumn
%\icmltitle{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\icmltitle{Robust and integrative Bayesian neural networks\\ for likelihood-free parameter inference\\
Supplementary Material}

\section{Extended Experiments for the MA(2) Test Problem:}
We investigate the effect of varying random seeds, and a comparison of execution times for the proposed approach against SNPE-C. We also explore the effect of varying the hyperparameters involved in the proposed BCNN-based approach.
\subsection{Qualitative evaluation with additional seeds}

\begin{figure*}[ht]
\begin{center}
\centerline{\includegraphics[width=\textwidth]{img/ma2_suppl_grid.pdf}}
\caption{Ten individual seeds for estimating posterior using SNPE-C (A) and BCNN (B). }
\label{fig:grid_LV}
\end{center}
\end{figure*}


\twocolumn
In order to investigate the robustness of the proposed approach, ten distinct random seeds were used in the parameter inference task for the BNN (BCNN) and SNPE-C based methods. Figure~\ref{fig:grid_LV} depicts the results with each row representing a seed, and each column representing a round in the sequential parameter inference process. The random walk Metropolis-Hastings MCMC method was used to calculate the exact posterior for the purpose of comparisons with BCNN and SPNE-C. 

It can be observed that the proposed BCNN-based approach is consistent across different seeds, while SNPE-C shows susceptibility in three cases (seeds 4,5 and 9). It can also be seen that SNPE-C typically starts with a tighter estimated posterior in the first round, but relative gains over subsequent rounds are small. In comparison, BCNN is slower in the initial stages, with larger estimated posteriors in the first round. However, relative gains over successive rounds are more significant with substantial refinement being achieved. This can be attributed to the exploitation offered by the binning-based classification approach involved in parameter inference using BCNN. The final estimated posterior in the sixth round is more concentrated for the BCNN as compared to SNPE-C. 

\subsection{Embedding Network architectures}
\label{sec:arch_ma2}
The BCNN was implemented using the TensorFlow library (Keras and tensorflow probability), while the SBI package (SNPE-C) makes use of PyTorch. The architectures are shown in Table \ref{tab:ma2}.
\begin{table}[ht]
\caption{Architecture for the BCNN (top) and SNPE-C (bottom); NDE refers to neural density estimator.}
\label{tab:ma2}
%\vskip 0.15in
\begin{center}
%\begin{small}
%\begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 100, 1)]&0\\
Conv1DFlipout (ReLU) & (None, 100, 6)  &66\\
MaxPooling1D    & (None, 10, 6)&0\\
Flatten    & (None, 60)&0\\
DenseFlipout (Softmax)     &(None, 16)&1936\\
\midrule
Total weights &&2002

\end{tabular}
%\end{sc}
%\end{small}
%\end{center}

%\caption{Architecture SNPE-C, icluding embedded summary statistics network and neural density estimator (NDE) with 10 hidden units and 2 transforms}
% \label{sample-table}
% \vskip 0.15in
% \begin{center}
% \begin{small}
% \begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 100, 1)]&0\\
Conv1d (ReLU) &(None, 100, 6)  &36\\
MaxPooling1d    &(None, 10, 6)&0\\
view (Flatten)    & (None, 60)&0\\
Linear (ReLU)     &(None, 16)&488\\
NDE (MAF) &10 hidden, 2 transform&970\\
\midrule
Total weights &&1494

\end{tabular}
%\end{sc}
%\end{small}
\end{center}
%\vskip -0.1in
\end{table}

\subsection{Hyperparameters of the BNN}
Here we experimented with different hyperparameters for the BNN (BCNN), including different number of bins (classes) in Figure~\ref{fig:ma2_bins}, and different exploitation strategies using the threshold $\delta$ in Figure~\ref{fig:ma2_thresh}. For MA(2), four bins per parameter (16 classes) is the best performing setting, however using five bins (25 classes) results in similar performance. For the exploitation threshold $\delta$, we observe the best overall performance using $\delta=0.05$ when observing log mean EMD. For mean MMD both show similar results. Additionally, a proposed strategy of using an exponentially decaying threshold, i.e., reducing the amount of exploitation per round, performs on a level in between $\delta = 0.05$ and $\delta = 0.0$. 
\begin{figure}[ht]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_bins__4.25x2_fontscale08.pdf}}
\caption{Log mean EMD and mean MMD for BCNN with different number of classes corresponding to the number of bins.}
\label{fig:ma2_bins}
\end{center}
\end{figure}

\begin{figure}[ht]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_thresh__4.25x2_fontscale08.pdf}}
\caption{Log mean EMD and mean MMD for BCNN for different threshold $\delta$. $\delta = 0.0$ refer to no threshold being used (i.e no exploitation). $\delta = 0.05$ is the strategy used in the main article. $\delta = e$ is an exponentially decaying threshold.}
\label{fig:ma2_thresh}
\end{center}
\end{figure}

\subsection{Execution time}
Figure~\ref{fig:ma2_time} shows the time difference for the inference task between the two implementations used in this study. Note that the BCNN used for MA(2) is one joint model (2D bins), however for the LV model several co-variate BCNN models are trained.

It is important to note that differences in software implementations (TensorFlow versus PyTorch) and approaches to parallelization make direct empirical comparisons challenging. Further, factors such as the early stopping criterion will also have an effect on the result.

%Note that this is by no means a final evaluation of the computational complexity between BCNN and SNPE-C. Both have different implementations and are using various parallel and neural network packages, further factors such as early stopping criterion will also have an effect on the result. However, Figure~\ref{fig:ma2_time} shows the time difference for the inference task between the two implementations used in this study. Observe that the BCNN used for MA(2) is one joint model (2D bins), however for the LV model several co-variate BCNN models are trained. 

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_time__4.25x2_fontscale08.pdf}}
\caption{(left) Time (minutes) per round between BCNN and SNPE-C using MAF. (right) Cumulative time (minutes). Both include simulation time per round (3000 simulations/round).  }
\label{fig:ma2_time}
\end{center}
\end{figure}

%\subsection{Various neural density estimators in SNPE-C}
%\begin{figure}[ht]
%\vskip 0.2in
%\begin{center}
%\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_NDE__4.25x2_fontscale08.pdf}}
%\caption{Text}
%\label{fig:ma2_time}
%\end{center}
%\vskip 0.2in
%\end{figure}




\section{Extended Experiments for the Lotka-Volterra (LV) Test Problem:}
The structure of extended experiments for the LV test problem follows the MA(2) setup in the text above. 
%We investigate the effect of varying random seeds, a comparison of execution times for the proposed approach against SNPE-C. We also explore the effect of varying the hyperparameters involved in the proposed BCNN-based approach.

\subsection{Qualitative evaluation with additional seeds}
%To investigate the robustness in the LFI-methods, we repeat the experiments multiple times, with five different seeds. 
The robustness of the three approaches - BCNN, SNPE-C and ABC-SMC against five different seeds is now investigated for the Lotka-Volterra predator-prey model.
In Figure~\ref{fig:lv_grid}, we show the snapshots and the final two-dimensional marginal distributions for the five independent runs. Comparison of the three approaches shows that the BCNN approach is relatively robust, while the ABC-SMC method converges slowly. It can also be seen that SNPE-C often converges to a different posterior shape as compared to BCNN and ABC-SMC. 



% \begin{figure}[ht]
% \vskip 0.2in
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_0.pdf}}
% \caption{Snapshot \# 1 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_0}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_1.pdf}}
% \caption{Snapshot \# 2 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_1}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_2.pdf}}
% \caption{Snapshot \# 3 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_2}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_3.pdf}}
% \caption{Snapshot \# 4 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_3}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_4.pdf}}
% \caption{Snapshot \# 5 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_4}
% \end{center}
% \vskip 0.2in
% \end{figure}

%\twocolumn
\subsection{Embedding Network architectures}
%The BCNN was implemented in TensorFlow (Keras and tensorflow probability) while the SBI package (SNPE-C) makes use of PyTorch. 
Implementations follow the description in Section \ref{sec:arch_ma2}. The architectures are shown in Table \ref{tab:lv}.
\begin{table}[ht]
\caption{Architecture BCNN (top) and SNPE-C(bottom).}
\label{tab:lv}
%\vskip 0.15in
\begin{center}
%\begin{small}
%\begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 51, 2)]&0\\
Conv1DFlipout (ReLU) &(None, 51, 20)  &260\\
MaxPooling1D    &(None, 10, 20)&0\\
Flatten    & (None, 200)&0\\
DenseFlipout (Softmax)     &(None, 25)&10025\\
\midrule
Total weights &&10285

\end{tabular}
%\end{sc}
%\end{small}
%\end{center}

%\caption{Architecture SNPE-C, icluding embedded summary statistics network and neural density estimator (NDE) with 10 hidden units and 2 transforms}
% \label{sample-table}
% \vskip 0.15in
% \begin{center}
% \begin{small}
% \begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 51, 2)]&0\\
Conv1d (ReLU) &(None, 51, 20)  &140\\
MaxPooling1d    &(None, 10, 20)&0\\
view (Flatten)    & (None, 200)&0\\
Linear (ReLU)     &(None, 25)&1608\\
NDE (MAF) &10 hidden, 2 transform&912\\
\midrule
Total weights &&2660

\end{tabular}
%\end{sc}
%\end{small}
\end{center}
%\vskip -0.1in
\end{table}

\subsection{Hyperparameters of BNN}
As in the case of the MA(2) test problem, the hyperparameters of interest to explore in the context of the BNN aproach for LV include the number of bins (classes) and the extent of exploitation controlled by the threshold $\delta$. In Figure~\ref{fig:lv_bins}, we explore three values for the bins. As the number of bins for the discretization of continuous data increases, so does the convergence speed towards a lower MSE.
%One could theorize that they all will converge to the same MSE, however, at different rates. 

In Figure~\ref{fig:lv_thresh}, we compare the MSE for 8 rounds using a threshold value of $\delta = 0.05$ and $\delta = 0$, i.e., with or without exploitation. It can be seen that incorporating exploitation via $\delta > 0$, results in a more rapid decline in MSE as compared to the case of no exploitation. This is expected, validates the purpose of exploitation in the proposed approach. 
%As in the case of the number of bins, the two cases might converge to the same MSE; however, when using exploitation $\delta > 0$, predictably, one reaches a lower MSE faster than without.

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_bins.pdf}}
\caption{MSE for BCNN with different number of classes corresponding to the square of the number of bins per parameter, on the LV test problem.}
\label{fig:lv_bins}
\end{center}
\end{figure}

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_thresh.pdf}}
\caption{MSE for BCNN when using threshold $\delta$: $0.05$ and $0.0$, on the LV test problem.}
\label{fig:lv_thresh}
\end{center}
\end{figure}

\subsection{Execution time}
Figure~\ref{fig:lv_time} presents the execution time of the BCNN and SNPE-C approaches. However, we note that several factors affect this comparison. It can be observed that the BCNN approach is slower as compared to SNPE-C owing to the overhead of training times for multiple classifiers.
%One should observe the execution time presented in Figure~\ref{fig:lv_time}  with caution. 
The number of required classifiers depends upon the number of parameter pair-combinations, e.g., three parameters requires four models, four parameters require six models, etc. The additional training time for the BCNN in case of LV can be reduced via better parallelization of the training process of the three classifiers. Please refer to Figure~\ref{fig:ma2_time} for a comparison of execution times for a single trained model.

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_time.pdf}}
\caption{Time per round (minutes) and cumulative time (minutes), for BCNN and SNPE-C, on the LV method.}
\label{fig:lv_time}
\end{center}
\end{figure}


\newpage
\onecolumn
%\begin{landscape}
\begin{sidewaysfigure}[ht]
\centering
\includegraphics[width=21cm]{img/LV_grid_seeds.pdf}
\caption{Snapshot \# 1-5 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
\label{fig:lv_grid}
\end{sidewaysfigure}
%\end{landscape}
\end{document}
toArxiv.tex
%%%%%%%% ICML 2021 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%%

\documentclass{article}

% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{booktabs} % for professional tables

% Added by authors
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{mathtools}


\usepackage{rotating}
\usepackage{pdflscape}



% hyperref makes hyperlinks in the resulting PDF.
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
% please comment out the following usepackage line and replace
% \usepackage{icml2021} with \usepackage[nohyperref]{icml2021} above.
\usepackage{hyperref}

% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Use the following line for the initial blind version submitted for review:
%\usepackage{icml2021}

% If accepted, instead use the following line for the camera-ready submission:
\usepackage[accepted]{icml2021_arxiv}

% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{Robust and integrative Bayesian neural networks for likelihood-free parameter inference}

\begin{document}

\twocolumn[
%\icmltitle{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\icmltitle{Robust and integrative Bayesian neural networks\\ for likelihood-free parameter inference}
% It is OKAY to include author information, even for blind
% submissions: the style file will automatically remove it for you
% unless you've provided the [accepted] option to the icml2021
% package.

% List of affiliations: The first argument should be a (short)
% identifier you will use later to specify author affiliations
% Academic affiliations should list Department, University, City, Region, Country
% Industry affiliations should list Company, City, Region, Country

% You can specify symbols, otherwise they are numbered in order.
% Ideally, you should not use this facility. Affiliations will be numbered
% in order of appearance and this is the preferred way.
\icmlsetsymbol{equal}{*}

\begin{icmlauthorlist}
\icmlauthor{Fredrik Wrede}{equal,uu}
\icmlauthor{Robin Eriksson}{equal,uu}
\icmlauthor{Richard Jiang}{ucsb}
\icmlauthor{Linda Petzold}{ucsb}
\icmlauthor{Stefan Engblom}{uu}
\icmlauthor{Andreas Hellander}{uu}
\icmlauthor{Prashant Singh}{umu}
\end{icmlauthorlist}

\icmlaffiliation{uu}{Department of Information Technology, Uppsala University, Uppsala, Sweden}
\icmlaffiliation{ucsb}{Department of Computer Science, University of California-Santa Barbara, Santa Barbara, CA, USA}
\icmlaffiliation{umu}{Department of Computing Science, Umeå University, Umeå, Sweden}

\icmlcorrespondingauthor{Fredrik Wrede}{fredrik.wrede@it.uu.se}
\icmlcorrespondingauthor{Prashant Singh}{prs@cs.umu.se}

% You may provide any keywords that you
% find helpful for describing your paper; these are used to populate
% the "keywords" metadata in the PDF but will not be shown in the document



\icmlkeywords{Bayesian Approximate Inference, Active Learning, Supervised Learning, Computational Biology, Bayesian Deep Learning}

\vskip 0.3in
]

% this must go after the closing bracket ] following \twocolumn[ ...

% This command actually creates the footnote in the first column
% listing the affiliations and the copyright notice.
% The command takes one argument, which is text to display at the start of the footnote.
% The \icmlEqualContribution command is standard text for equal contribution.
% Remove it (just {}) if you do not need this facility.

%\printAffiliationsAndNotice{}  % leave blank if no need to mention equal contribution
\printAffiliationsAndNotice{\icmlEqualContribution} % otherwise use the standard text.
\begin{abstract}
State-of-the-art neural network-based methods for learning summary statistics have delivered promising results for simulation-based likelihood-free parameter inference.
Existing approaches for learning summarizing networks are based on deterministic neural networks, and do not take network prediction uncertainty into account. 
This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and produces a proposal posterior density using categorical distributions. 
An adaptive sampling scheme selects simulation locations to efficiently and iteratively refine the predictive proposal posterior of the network conditioned on observations.
This allows for more efficient and robust convergence on comparatively large prior spaces. 
The approximated proposal posterior can then either be processed through a correction mechanism to arrive at the true posterior, or be used in conjunction with a density estimator to obtain the posterior.
We demonstrate our approach on benchmark examples.% and compare against related methods.
% Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space.
% % ** 1 **
% Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account. 
% This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions.
% %This work proposes an uncertainty-enhanced summary statistics learning framework using Bayesian neural networks, which incorporates epistemic uncertainties to quantify the confidence of the trained model. 
% % ** 2 **
% An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network.
% %conditioned on observations, hence eliminating the need for discrepancy-based rejection or density estimation and reducing the number of simulations required. 
% % ** 3 **
% %Our novel approach also includes estimating the posterior using categorical distributions, subsequently transforming the learning task into a classification problem. 
% This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.
\end{abstract}

\section{Introduction}
\label{sec:intro}
 Likelihood-free inference (LFI) is a class of methods designed for parameter inference of models with intractable or expensive likelihoods. 
 %The type of models range from ... 
 Approximations to the posterior distribution conditioned on observed data are made with the aid of forward simulations of the model, thus also referred to as simulation-based inference~\cite{Cranmer_frontierSBI}. Approximate Bayesian computation (ABC)~\cite{sisson2018handbook} is a popular method in this setting, and involves the use of a discrepancy measure to compare simulated data against observed data. A threshold on the discrepancy measure decides whether a proposed parameter can be accepted as a sample from the posterior. 
 
While this approach has shown to be robust, it has certain drawbacks. For many dynamical models, the simulation output is high-dimensional, 
and one uses summary statistics as a dimension reduction technique
 % avoid large approximation errors
 \cite{sisson2018handbook,wood2010statistical,engblom2020bayesian}. Selecting informative summary statistics so that the discrepancy measure can accurately accept posterior samples is often a tedious task, typically requiring detailed domain knowledge together with a trial and error approach. 
 
 An alternative approach to handpicking summary statistics is to utilize embedded neural networks (NN) to learn summary statistics from simulated data \cite{jiang2017learning, dinev2018dynamic, wiqvist2019partially, aakesson2020convolutional,chen2020neural}.  Some of these methods require a post-processing ABC scheme to yield the posterior, others work in conjunction with neural density estimators \cite{lueckmann2021benchmarking} to either estimate the posterior directly, or use synthesized likelihoods which requires Markov chain Monte Carlo (MCMC) sampling. Although deterministic NNs for learning summary statistics have shown impressive results, they are prone to be inaccurate in regions where the training data is sparse or not well represented (particularly in the case of problems with high-data dimensionality) \cite{aakesson2020convolutional}. 
 %The uncertainty of prediction or network prediction confidence is not taken into account in the LFI pipelines.
 %they can often lead to too `overconfident' predictions\textcolor{red}{cite}. Although implementations of embedding networks include regularization techniques to avoid overfitting, they still suffer from inaccuracies in regions of the parameter space not well-represented in the training data. 
Deterministic NNs do not directly provide an estimate of their prediction uncertainty, which can potentially mislead the parameter inference process.
%Deterministic NNs cannot quantify uncertainties associated with unseen data independently, which manifests as an inherent problem when one samples a finite amount of parameter proposals from large, potentially uninformative priors. 
This lack of direct uncertainty quantification motivates the design of more robust and expressive embedding networks, that can drive the LFI process in a more reliable manner.
 
 % ToDo for PS: elaborate more on the overconfident part - this is due to (limited) sampling in hd spaces, with large gaps of no data
 % PS: Perhaps we should clearly say - Approach 1: (BNN) and Approach 2: (BNN + classification)
%This paper presents a novel likelihood-free parameter inference framework that improves upon two characteristics of embedded NN-based LFI methods, as outlined below.

Bayesian embedded NNs are proposed as summary statistics that provide prediction uncertainties, and coupled with a distribution estimator, also approximate the proposal posterior. An adaptive sampling algorithm utilizes these uncertainty estimates to iteratively refine the approximated proposal posterior. Including prediction uncertainty in the LFI process provides robustness against potential gaps in training data.

The true posterior distribution can be obtained from a correction procedure applied to the proposal posterior (out of scope of this work). The need for the correction arises due to the iterative nature of the adaptive sampling algorithm. The proposal posterior is refined in each iteration by sampling solely within the proposal posterior of the previous iteration. Therefore, in order to account for \emph{proposal leakage} or regions within the true posterior missed due to inaccuracies in the NN, a correction mechanism is needed that enlarges the proposal posterior accordingly.

%This work explores Bayesian embedded networks as trainable summary statistics that provide prediction uncertainties, and directly estimate the posterior distribution.
%Introducing epistemic uncertainties in the embedded network will enable a more robust LFI workflow by considering model confidence in addition to model predictions. The embedded network's predictive posterior will directly estimate the posterior distribution for the inference task, ultimately integrating embedded summary statistics networks with neural density estimators. 
The proposed approach involves approximating the proposal posterior through discrete categorical distributions, subsequently solving a regression problem via classification. This approach in similar form has delivered encouraging results~\cite{weiss1995rule, torgo1996regression, AHMAD_RvC}. The main motivation behind this design is that performing regression with high-dimensional inputs (such as dynamical simulation responses) to continuous multi-dimensional outputs (such as parameters) is a highly challenging task. Even though NNs have the learning capacity to do so, generalizing the networks often proves difficult. 
%Particularly if the training data is not fixed over individual training procedures, such as the case for parameter inference. It's our intention to make parameter inference using NNs more robust by utilizing regression via classification.     

 We demonstrate the efficiency and robustness of our approach on benchmark problems, and also compare the proposal posteriors against approximated posteriors obtained using deterministic NNs paired with neural density estimators. Experimental results show  that the proposed approach is consistently more robust than existing methods, with relatively small variance in parameter inference quality performance. The error in parameter estimation even without the correction process is comparable to existing methods. %\textcolor{red}{Think more about how we add value over SNPE}
 %As an alternative approach, some methods have been focusing on introducing inverse mapping functions $f$.  By fitting the inverse mapping function $f(\bold{y}, w) \mapsto \hat{\theta}$ using simulated data  $X = \{y_i, \theta_i\}^N$, a loss function $L(\theta_i, \hat{\theta_i})$ can be optimized w.r.t $w$ between predicted and true parameters, the weights $w$ should ultimately incorporate the discrepancy between samples. While some variants can directly yield density estimators $f(y_0,w) \mapsto \hat{p}(\theta|y_0}$, others use it in a post-processing rejection scheme by letting $d(f(y_i,w), f(y_0,w))$. Learning the inverse mapping function is typically referred as a regression problem where one of the most popular approaches is to use artificial neural networks (ANN).}

%However, most applied deep learning architectures are deterministic and non-probabilistic and can yield overconfident predictions on targets. Further, classification as shown to be superior to regression in prediction performance when $f$ inherits complex non-linear patterns. Thereof, we explore using Bayesian neural networks to better account for epistemic uncertainty and regularization in the mapping function and we discretize $\theta$ space into categorical classes to transform the problem into a classification task, assuming that sample paths $y$ to be similar when in parameter proximity. The inherent properties of dynamical processes makes convolutional neural networks (CNN) an attractive option for learning the mapping function $f$. CNNs make use of a filtering kernel to create local feature maps.Previous studies have demonstrated the capabilities of using determinstic CNNs for likelihood-free parameter inference \cite{}. In this study we use a similar architecture but make use of probablistic layers in Tensflow probability package to place parameterized distributions over the weights of the neural network and apply variational inference using evidence lower bound (ELBO) and \textit{Bayes by Backprop} to fit a posterior distribution. This approach suggests that, not one, but an ensemble of neural networks are learned, where each network's weights are sampled from the fitted posterior. While it might seem computationally heavy learning an infinite ensemble of networks, in fact the amount of weights only increases by a factor of two when using Monto Carlo estimates of the gradients on an otherwise intractable problem.
We organize the paper as follows: Section~\ref{sec:background} introduces the problem setting, the notation, and related work. Section~\ref{sec:method} presents the proposed LFI parameter estimation approach. Section~\ref{sec:experiments} demonstrates the proposed methodology on benchmark examples and draws comparisons to other methods. Section~\ref{sec:discussion} presents a discussion on the merits and applicability of the approach. Finally, Section~\ref{sec:conclusion} gives concluding remarks about the novel method presented in the paper.

\section{Background} %  notation, estimate PDF with Neural net (with embedding net)
\label{sec:background}
% problem set-up | we want the posterior given model M, and observation y0
Let ${\bf y}_0$ be a target observation which can be approximately modeled by $M({\bf y} | {\boldsymbol \theta})$.
%Assume a dynamical model  $\mathcal{M}$ which has generated an observation ${\bf y}_0$. Next, assume some model error by approximating the model by $M({\bf y} | {\boldsymbol \theta})$, with parameters $\boldsymbol \theta$.
Forward simulation of the model $M$ results in the sample path ${\bf y}^{1:T}$ henceforth referred to as ${\bf y}$.
%Let $ {\bf y}_0 \sim M({\bf y}| {\boldsymbol \theta})$ be observed data approximately sampled from $M({\bf y}|\boldsymbol{\theta})$, i.e., we assume some model error.
%The observed data may also have been obtained from the physical process corresponding to $M$. 
The objective is to infer the parameters conditioned on the observation, i.e., the posterior density $p(\boldsymbol{\theta} | {\bf y}_0) \propto p({\bf y}_0 | \boldsymbol{\theta})p(\boldsymbol{\theta})$. However, as we assume that the likelihood is computationally intractable, which is often the case for dynamical models, an estimate $\hat{p}(\boldsymbol{\theta} | {\bf y}_0)$ of the true posterior distribution must be approximated. Approaches in this setting typically utilize access to the observed data  ${\bf y}_0$ and the ability to perform simulations from model $M$ at well-chosen parameter locations. We summarize some the methodologies under the supposed setting below.
%, and motivate the need for a fully Bayesian model-driven LFI approach.


 
 

\subsection{Related work}
% --- Simulation based Inference --- %
% discrepancy-, SS based
The ABC family of methods forms the most popular and established LFI approach (introduced in Section~\ref{sec:intro}). 
%The methodology consists of a sample-simulate-compare cycle where parameter samples $\boldsymbol{\theta}_i \sim p(\boldsymbol{\theta})$ drawn from a defined prior distribution $p(\boldsymbol{\theta})$ are simulated using the model $M$ to obtain sample paths ${\bf y}$, which one consequently compare to observed data ${\bf y}_0$ to either declare T1 as being accepted as part of the estimated posterior or rejected. The workflow consists of several hyperparameters, including the choice of the prior P1, dimension reducing summary statistics (when ${\bf y}$ is high dimensional, e.g., time series), a distance or discrepancy measure to compare simulated and observed data, and a tolerance bound $\epsilon$ to define acceptance/rejection. 
Although remarkably successful, ABC methods are typically challenging to setup optimally for accurate inference and convergence speed \cite{sisson2018handbook,marin2012approximate}. Extensions such as sequential Monte Carlo methods (ABC-SMC) \cite{sisson2007sequential} can potentially accelerate the convergence speed, but still require laborious hyperparameter selection. In particular, the choice of informative summary statistics, is still present, and is critical for practical and accurate parameter inference as it forms the means of comparing simulated proposals with the observation. 

Consequently, summary statistic selection has received considerable attention \cite{sisson2018handbook}[Chapter 5]. However, traditional summary statistics such as statistical moments may not always be expressive enough to capture intricate details in rich dynamical data. Further, ideal statistics may even be missing from candidate pool of statistics to select from, e.g., \emph{tsfresh}~\cite{christ2018time}. A recent development has been the use of data-driven regression models to extract features from data, and function as summary statistics. It has been shown that the ideal summary statistic is the posterior mean, when minimizing the quadratic loss \cite{fearnhead2012constructing}. In particular, NN based regression models have delivered highly promising results towards learning the estimated posterior mean as summary statistics from high-dimensional detailed time series \cite{jiang2017learning,wiqvist2019partially,aakesson2020convolutional}. Such models learn the mapping $F_{\bf w}({\bf y}) \mapsto \mathbb{E}[\boldsymbol{\theta}]$ which can be seen as an amortized point estimate for a given ${\bf y}$, where ${\bf w}$ are the weights of the regression model. However, an overhead associated with \emph{trainable} summary statistics is the need for large amounts of simulated training data to yield an accurate  posterior mean estimate. Although there are opportunities for re-using the training data in the later rejection sampling step to estimate the conditioned posterior, the sheer amount of training data required is problematic whenever the simulations are expensive to compute. % Mention that they can't quantify uncertainties about their estimates 

%Synthetic likelihood based approaches \cite{wood2010statistical}  step away from discrepancy distances and approximate the likelihood conditioned on the summary statistics.

% remove SS, still discrepancy-dependent
%Efforts to use embedding nets to compute optimal summarizing statistics have shown great promise, NN \cite{jiang2017learning}, Convolutional NN \cite{aakesson2020convolutional}, but they still necessitate a discrepancy distance to evaluate proposals. These approaches consists of fitting a regression model $F_w(y) \mapsto \theta$ parameterized by the weights $w$ and subsequently use prediction on simulated data
%\textcolor{red}{FW: need to introduce SNPE-C in particular\cite{greenberg2019automatic}}
Neural density estimators form a family of methods that directly estimate densities with the aid of NNs and have shown impressive performance on challenging problems \cite{lueckmann2021benchmarking}, such as the masked autoregressive flows (MAF) and the neural spline flow (NSF) \cite{papamakarios2017masked,NSF}. While some neural density estimators are used to estimate posterior densities \cite{APT_2019,SNPE_A_Papamakarios, SNPE_B_lueckmann2017flexible}, others rely on constructing synthesized likelihoods or ratios which require MCMC sampling to extract the posterior \cite{papamakarios2019sequential, lueckmann2019likelihood,SNRE_a, SNRE_b_constrative}. The formed posteriors are amortized, i.e., any observation can be conditioned on the posterior. 

However, a collection of algorithms based on neural density estimators uses sequential Bayesian updates to the posterior, conditioned on a particular target observation. These include sequential neural posterior estimation (SNPE) \cite{SNPE_A_Papamakarios, SNPE_B_lueckmann2017flexible, APT_2019},  sequential neural likelihood estimation (SNLE) \cite{papamakarios2019sequential, lueckmann2019likelihood} and sequential neural ratio estimation (SNRE) \cite{SNRE_a, SNRE_b_constrative}. A benchmarking effort was recently conducted comparing the performance of these approaches on various problems \cite{lueckmann2021benchmarking}. These approaches also support the use of embedding networks to turn high-dimensional simulation outputs to informative summary statistics. The weights of the embedding networks are then trained together with the weights of the neural density estimator. Thus, these shared architectures enable sequential updates on the conditioned posterior and the summary statistics learning process, ultimately reducing the amount of simulations required. 

SNPE-C (also known as automatic posterior transform - APT) \cite{APT_2019} is a recent SNPE technique that learns to infer the true posterior by maximizing an intermediate estimated proposal posterior. SNPE approaches form a good class of methods to compare the proposed approach to, as the use of sequential Bayesian updates to refine the estimated posterior is close in spirit to adaptive sampling used in this work. We note that SNPE approaches approximate the true posterior, while the proposed approach needs a correction step as post-processing to arrive at the true posterior from a proposal posterior. SNPE approaches have also been shown to perform well across a variety of test problems \cite{lueckmann2021benchmarking,APT_2019}. Among SNPE approaches, SNPE-C is superior as it is more flexible and scalable as compared to SNPE-A and SNPE-B \cite{APT_2019}, and is used for the purpose of comparisons in the experiments in this paper. 


%The most recent and superior version of SNPE is the Automatic Posterior Transform (APT) \cite{APT_2019}, referred as SNPE-C in the software package SBI\footnote{https://www.mackelab.org/sbi/} which is the naming we will be using henceforth.       
    
% A list, that then has to be made into a flowing text.
% *classification*
%Using classification in LFI is not novel~\cite{gutmann2018likelihood}.  
%
% *flow-based*
%Flow-based model $q(\eta|x)$ condition on $x$ to generate the next training data set $\mathcal{D}$ in order to approximate the posterior $p(\eta|x)$, APT (SNPE-C) \cite{greenberg2019automatic, gonccalves2020training}
%Flow-based model with $q(\eta|x)$ conditioned on $\eta$ \cite{papamakarios2019sequential}
%SNPE \cite{lueckmann2017flexible,lueckmann2019likelihood,papamakarios2017masked}
% --- collecting ---
%Benchmarking effort of SBI \cite{lueckmann2021benchmarking}, 

\subsection{Motivation for Bayesian neural networks}
We implement a NN for summary statistics, with density estimation through Bayesian neural networks (BNNs). The introduction of weight uncertainties in NNs has enabled a robust means for learning an ensemble of models \cite{Bayesbybackprop}, represented by the posterior density $p({\bf w}|{\bf y})$ where ${\bf w}$ are the network weights.
%Blundell \textit{et al.} demonstrate with their Bayes by backprop algorithm how one can obtain and fit an ensemble of models represented by the posterior density $p(w|X)$ where $w$ is the network weights and $X$ the training data. 
It has also been demonstrated \cite{Bayesbybackprop} that the number of weights only increases by a factor of two by utilizing Monto Carlo (MC) sampling of gradients when updating the weights. The benefit of using a BNN is that the proposal posterior predictive distribution $\tilde{p}_w(\boldsymbol{\theta}|{\bf y}) = \int \tilde{p}(\boldsymbol{\theta}|{\bf w}, {\bf y})p({\bf w}|{\bf y})d{\bf w}$ provides an uncertainty measure to its predictions. The uncertainty estimates form an elegant means of identifying and countering overfitting, which translates into more robust and efficient embedding networks. Different embedding network architectures such as RNNs or CNNs can be implemented in the BNN setting to provide prediction uncertainty estimates, and compute the proposal posterior $\tilde{p}(\boldsymbol{\theta}|{\bf y}_0) \coloneqq \tilde{p}_w(\boldsymbol{\theta}|{\bf y}_0)$ by the proposal posterior predictive distribution. We will henceforth denote $\tilde{p}_w(\boldsymbol{\theta}|{\bf y})$ simply as $\tilde{p}(\boldsymbol{\theta}|{\bf y})$. The uncertainty estimates also allow sequential adaptive sampling of the posterior, conditioned on a target observation for faster convergence, similar to SNPE. 

%However, due to the integral in the predictive posterior being analytically intractable, numerical integration needs to be utilized (e.g., using MC methods). For the application of LFI, a BNN will provide the flexibility of using embedding networks, e.g., architectures such as RNNs or CNNs, that provide uncertainty estimates of predictions, and directly estimate the posterior $p(\theta|y_0) \approx p(\theta|X)$ by the posterior predictive distribution. This, in turn will enable sequential adaptive sampling of the posterior conditioned on a target observation for faster convergence. 
%\textcolor{red}{FW: Not completely sure of the notation of posterior predictive distribution in our case is it $p(\theta|X)$, $p(y_0|X)$, $p(\theta|w,X)$ etc??} 
%\textcolor{blue}{PS: $p(\theta|X)$, or $p(\theta|w,X)$ (there are arguments on both sides whether or not to include $w$. I'd go with $p(\theta|X)$. It is okay to abstract model specifics (in the form of $w$).}
%The need for large quantities of (potentially computationally expensive) simulated data for training summary statistics and subsequently estimating the posterior distribution using the ABC family of methods can lead to impractical parameter inference processing times. Therefore, this paper aims at developing a data-efficient LFI parameter estimation pipeline that minimizes the training data needed to obtain accurate high-quality summary statistics using an adaptive sampling algorithm, and directly estimates the posterior using Bayesian neural networks thereby eliminating the need for ABC-based posterior estimation. Further, an efficient classification-based approach is also presented that iteratively shrinks the prior search space.
% 
% 

\section{Embedded Summary Statistics via Bayesian neural networks}
\label{sec:method}
 Let $D = \{({\bf y}_i,\boldsymbol{\theta}_i)\}_1^N$ consist of simulated data and parameter proposals sampled from the prior distribution over the parameter space $p(\boldsymbol{\theta})$. Consider a BNN, parameterized by the weights ${\bf w}$, that computes the posterior density $p({\bf w}|{\bf y})$ of a probabilistic model $\tilde{p}(\boldsymbol{\theta}|{\bf y},{\bf w})$ by placing a prior $p({\bf w})$ on the weights. The proposal posterior predictive distribution for new unseen sample path ${\bf y}_t$,  $\tilde{p}(\hat{\boldsymbol{\theta}}|{\bf y}_t) = \mathbb{E}_{p({\bf w}|D)}[\tilde{p}(\hat{\boldsymbol{\theta}}|{\bf y}_t, {\bf w})]$ is the expectation on unseen data. Setting ${\bf y}_t = {\bf y}_0$, for $\hat{\boldsymbol{\theta}} \approx \boldsymbol{\theta}$ the proposal posterior predictive distribution will be equal to $\tilde{p}(\boldsymbol{\theta}|{\bf y}_0)$ as $N \rightarrow \infty$, which is the target parameter proposal posterior conditioned on a particular observation. 
 %Additionally, when the probabilistic model $p(\boldsymbol{\theta}|{\bf y},{\bf w})$ is a regression model, it also will yield $\hat{\boldsymbol{\theta}} = \boldsymbol{\theta}$. However, 
 As we demonstrate next, we will approximate the parameters $\boldsymbol{\theta}$ through discrete categories making ${\tilde p}(\boldsymbol{\theta}|{\bf y},{\bf w})$ a categorical distribution thereby transforming the regression problem into a classification problem. 

\subsection{Probabilistic regression via classification}
Let $\tilde{p}(\boldsymbol{\theta}'|{\bf y},{\bf w})$ be represented as a categorical distribution using cross-entropy and softmax activation in a NN classifier denoted by $F_{\tilde{p}(\boldsymbol{\theta}'|{\bf y},{\bf w})}$.  We will transform the continuous problem by discretizing $\boldsymbol{\theta} \in \mathbb{R}$ into $\boldsymbol{\theta}'\in K$ by $B$ equal width bins per dimension (please see Section~\ref{sec:discussion} for a discussion on binning strategies), i.e., the predictive distribution will provide the probability of belonging to a particular bin $K \in {1,2,...,B}$. By placing a multivariate Gaussian on $p({\bf w})$ we will fit $p({\bf w}|D)$ using Bayes by backprop, subsequently yielding the BNN classifier  $F_{p({\bf w}|D)}$. To estimate the proposal posterior predictive distribution conditioned on our target observation, we sample from $F_{p({\bf w}|D)}$ using a MC sampling scheme,   
\begin{equation}
    \tilde{p}(\boldsymbol{\theta}'|{\bf y}_0) \approx \frac{1}{N}\sum_{i=1}^N F_{p({\bf w}|D)}({\bf y}_0, {\bf w}_i).
\end{equation}

To enable sequential adaptive sampling during the inference task, we will train $G$ subsequent classifiers $F^g_{p({\bf w}|D)}, \, g = 1, \dots, G$, where the bins are updated by sampling from the proposal posterior predictive distribution. Each iteration employs the density from the previous generation as the prior for sampling new training data. The BNN generates amortized proposal posterior estimates. However, when conditioned on the observation, it allows for adaptive refinement in each iteration. To be able to simulate new samples, we need to transform $\boldsymbol{\theta}'$ back to $\boldsymbol{\theta}$. This is achieved by transforming each bin $K$ into a uniform distribution,

\begin{equation}
\begin{aligned}
    \{\boldsymbol{\theta}'\}_{i=1}^N &\sim \tilde{p}(\boldsymbol{\theta}'|{\bf y}_0) \\
    \{\boldsymbol{\theta}\}_{i=1}^{N} &= \{\mathbb{U}^K({\boldsymbol{\theta}'}_i^{\text{\emph{left}}},{\boldsymbol{\theta}'}_i^{\text{\emph{right}}})\}_{i=1}^N,
\end{aligned}
\end{equation}

where \emph{left} and \emph{right} are the boundaries of the bin intervals. The adaptive sampling-driven BNN is depicted in Figure~\ref{fig:pipeline} and detailed in Algorithm~\ref{alg:posterior}.

\begin{figure}[t]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/pipeline_v3.pdf}}
\caption{Adaptive sampling-driven BNNs. (A) Parameter proposals are sampled from a prior and simulated via the model. (B) Regression via classification. Continuous proposals are discretized into a categorical distribution with classes being the bins. A BNN (having prior weights) is then trained on data. (C) The posterior predictive conditioned on observation $y_0$ can be estimated via MC sampling, and is subsequently used in an iterative procedure, replacing the previous prior. Note that a transformation is applied to the categorical distribution to transform proposals back to continuous.}
\label{fig:pipeline}
\end{center}
\vskip 0.2in
\end{figure}

\subsection{Exploitation of bins}
%With large priors $p(\boldsymbol{\theta})$, finding the subset $p(\boldsymbol{\theta}|{\bf y}_0)$ can be a slow process.% 
We aim to exploit the uncertainty estimates provided by the proposal posterior predictive distribution to enable faster convergence. A key reason why the process can be slow is that low probability weights linger between iteration updates. We exploit the proposal posterior predictive distribution by using a threshold value, $\delta$. Given the categorical distribution, we set the classes with probability weight below the threshold to zero. The non-zero probabilities are then normalized, constructing a less skewed proposal distribution. Between iterations, $\delta_i$ can be chosen adaptively or kept constant throughout. We recognize that the sequence of distinct $\delta_i$ holds a similarity to the discrepancy thresholds ($\epsilon_i$) used by ABC-SMC~\cite{sisson2007sequential, del2012adaptive}.
% ** if not adaptive **
The potential adaptive nature is briefly discussed in Section~\ref{sec:discussion}.
%% *** if we decide on using adaptive ***
%To slow the exploitation, we employ an $\epsilon$ that converges to zero as the rounds increase.



\subsection{Dealing with the curse of dimensionality}
For the single-parameter problem, the structure of the classification problem is straightforward: one classifier for one parameter. However, for multiple parameter dimensionality, we construct one classifier for each two-dimensional covariance combination: $(\boldsymbol{\theta}_i, \boldsymbol{\theta}_j)$, implying that for a model of $k$ parameters, we would require $\frac{k!}{(k-2)!2!}$ classifiers. One sees that the number of classifiers quickly increases; however, they lend themselves towards an embarrassingly parallel workflow since they are independent, and use the same simulation data. Further, the adaptation of the proposals is done per covariance dimension and then marginalized, resulting in the $k$-dimensional parameter proposals.


%The LFI problem is solved by finding the inverse map $F(y)\mapsto \theta$. Consider the approximation $F_w(y)$ produced by an embedded net with weights, $w$. By fitting the inverse mapping function $F_w(y)$ using simulated data $\mathcal{D} = \{(y_i,\theta_i)\}_1^N$, a loss function $\mathcal{L}(\theta_i, \hat{\theta_i})$ can be optimized w.r.t. $w$ between predicted and true parameters; the weights $w$ should ultimately incorporate the discrepancy between samples.
%Our probabilistic approach allows incorporating epistemic uncertainty by employing Bayesian embedding nets (BNN) \cite{mackay1992practical}, which learn distributions of $q(w|\mathcal{D})$ instead of a point estimate $\hat{\theta}$ by introducing a prior $p(w)$ on the weights. Approximating the predictive posterior $p(y_0 |\mathcal{D})$ by Monte Carlo integration gives an interpretation of $\hat{p}(\theta | y_0)$, namely approximating the integral $ p(y_0 |\mathcal{D}))  = \int p(y_0 |\mathcal{D}),w)q(w|\mathcal{D}))dw$
%by a sum of independent samples $ p(y_0 |\mathcal{D})) \approx \frac{1}{M}\sum_{i=1}^M p(y_0 | w_i)$
%where $w_i \sim q(w | \mathcal{D}))$. 
%A connection between the NN and the BNN is that NN's weights can be seen as maximum likelihood estimates or maximum a posteriori estimates, when using regularization. %\cite{}? could use, https://arxiv.org/pdf/2007.06823.pdf but not sure how close they are to be published results.

%\begin{equation}
%    \begin{aligned}
%        p(y_0 |\mathcal{D}))  = \int p(y_0 |\mathcal{D}),w)q(w|\mathcal{D}))dw\\
%         \approx \frac{1}{M}\sum_{i=1}^M p(y_0 | w_i)\\
%         \approx \hat{p}(\theta|y_0)\\
%         \text{where } w_i \sim q(w | \mathcal{D}))
%    \end{aligned}
%\end{equation}


% The classification problem
%\subsection{Accelerating posterior estimation using classification}
%To achieve robust results (\textcolor{red}{FW: this is not enough motivation for using classification}), around the sought after posterior distribution $\hat{p}(\theta|y_0)$, we consider a sequence of discrete mappings from $y \mapsto \theta$ with increasing resolutions. Therefore, we solve finding the continuous map $F$ around $y_0$ by a series of approximate mappings, i.e., $F = f_1 \circ f_2 \circ \dots \circ f_G$. 
%The discrete map $f_k$ considers binned continuous data, meaning, finding $f_k$ is a classification problem for a categorical distribution. We condition each iterative map on $y_0$ to determine the area of interest in the prior, i.e., $f_{k+1}$ follows $f_k$, and $P_{f_k}(\theta|Y_0)$ gives the samples of interest. 
% The final inference step
%A key feature inherent of BNN is that the map $F$ direcred}tly gives the probability $p(\theta|y)$, and given the observation $y_0$, we get $\hat{p}(\theta|y_0)$ without any further steps.


%\section{Classification for LFI using Bayesian neural networks}
%Our approach's contribution is that the BNN allows for uncertainty estimates directly while on the classification step, instead of using the accuracy as a pseudo-measure for the discrepancy between the observation and simulation. We construct the classes by grouping continuous-data into a fixed number of bins. As the number of bins approach the number of samples, we approach the continuous, regression case.  

%The posterior density estimation by BNN is generated iteratively by a sequence of generation densities.  Each iteration employs the previous generation density as it's prior for sampling new training data. The BNN generates amortized posterior estimates. However, when conditioned on the observation, it allows for adaptive refinement in each iteration.

%Denote the trained BNN as $F(y|w)$ with weights $w$, the posterior distribution, approximated by the BNN, $q_{F(y,w)}$. We generate $q_F$ by Monte Carlo-sampling the network, and retrieve a categorical distribution over the binned data.


\begin{algorithm}[]
   \caption{Iterative BNN proposal posterior estimation }
   \label{alg:posterior}
\begin{algorithmic}
   \STATE {\bfseries Input:} model simulator $M({\bf y}|\boldsymbol{\theta})$, 
   observations ${\bf y}_0$, Bayesian neural network $F_{p({\bf w}|D)}$, 
   number of generations $G$, number of simulations per generation $N$,
   prior distribution $p(\boldsymbol{\theta})$, number of bins $B$.

   \STATE $p_0 := p$  
   \FOR{$g=1$ {\bfseries to} $G$}
        %\FOR{$n=1$ {\bfseries to} $N$} % training
            \STATE Sample $\boldsymbol{\theta}_N = \{\boldsymbol{\theta}_i\}_{i=1}^N$ from $p_{g-1}(\boldsymbol{\theta})$
            \STATE Sample ${\bf y}_N = \{{\bf y}_i\}_{i=1}^N$ from $M({\bf y}|\boldsymbol{\theta}_i)$
       % \ENDFOR
        \STATE Group $\boldsymbol{\theta}_N$ into K bins $\boldsymbol{\theta}'_N, \, K= 1:B$
        \STATE Train $F^g_{p({\bf w}|D_g)}$ on $D_g = \{{\bf y}_N, \boldsymbol{\theta}'_N\}$
        \STATE Estimate $\tilde{p}_{g}(\boldsymbol{\theta}'_N|{\bf y}_0) \leftarrow \mathbb{E}[F^g_{p({\bf w}|D_g)}({\bf y}_0,{\bf w}_N)]$
        \STATE Transform $p_g(\boldsymbol{\theta}) \leftarrow \mathbb{U}^K({\tilde{p}}_{g}(\boldsymbol{\theta}'_N|{\bf y}_0))$
        %\STATE \textcolor{red}{FW: Not sure what notation to use for the transform}
    \ENDFOR
    \STATE {\bfseries return} $p_g(\boldsymbol{\theta})$
\end{algorithmic}
\end{algorithm}

%\subsection{Complexity}
%\textcolor{red}{FW: Can we say something more about the complexity of the approach more than the number of weights? E.g simulations $>>$ training time? Also curse of dim, several classifiers are embarrassingly parallel and the number of simulations does not increase. Very important that the reader understand this.} 

%\subsection{Design of the BNN}
%\textcolor{red}{FW: I think this part should go into experiments as a particular case of the BNN tailored for dynamic models}
%In this work we have implemented the BNN as Convolutional NN, henceforth called BCNN. However, we would to point out that many other types of architectures can be used. Out motivation behind this choice is that CNNs have shown great performace in deducting inormation from time-series data \cite{sadouk2019cnn, aakesson2020convolutional}, and not only for images. To turn the layers of a CNN into probabilitic layes we have used the flipout \cite{wen2018flipout} layes supported in Tensorflow \cite{tensorflow2015-whitepaper} probability package. 

%\subsection{Adaptive sampling}


\section{Experiments}
\label{sec:experiments}
%\textcolor{red}{FW: missing transition into CNN (BCNN), have not decided if it should be introduced in section \ref{sec:method} or as a particular case of a BNN in section \ref{sec:experiments}}

To test our approach we have implemented a convolutional BNN (we denoted this particular implementation BCNN). However, we note that our approach is not restricted to these types of architectures.  Our motivation behind choosing a convolutional architecture is that they have demonstrated excellent performance in identifying intricate patterns in complex, high-dimensional time-series data \cite{dinev2018dynamic, sadouk2019cnn, aakesson2020convolutional} as well as images~\cite{APT_2019}. For further information on the implementation we refer the reader to supplementary materials.  


We explore the performance of the BCNN on two benchmark examples - the moving averages model (MA) and the discrete-state continuous-time Markov chain implementation of the Lotka-Volterra (LV) \cite{lotka}. Together with the resulting proposal posterior we recover using the BCNN, we also compare the proposal posterior estimation quality to posterior estimates obtained using ABC-SMC and SNPE-C with the MAF as the neural density estimator.

The SBI package \cite{tejero2020sbi} implements SNPE-C, and is used for the experiments in this paper. To keep the comparison fair between SNPE-C and the BCNN, we designed the embedded nets to be as similar as possible between the two approaches, resulting in having approximately the same number of weights in the two models.  
 
All of the experiments were conducted on a machine consisting of a 12-core Intel Xeon E5-1650 v3 processor operating at 3.5 GHz, 64 GB RAM, running Fedora core 32 Linux operating system. Even though GPU-acceleration would speed up NN training substantially, in the considered setting we assume that simulations are significantly more expensive than training time. This is the case for the discrete and stochastic LV model, less so for MA.   


\subsection{Moving average model}%- Noisy, high data dimensionality
% \begin{equation}
%     X_t = \epsilon_t + \theta_1\epsilon_{t-1} + \theta_2\epsilon_{t-2},
%     \epsilon_t \sim N(0,1) 
% \end{equation}
The moving average (MA) model is a popular benchmark problem in the ABC setting \cite{marin2012approximate}. The MA model of order $q$, MA($q$) is defined for observations $X_1, \dots, X_p$ as \cite{wiqvist2019partially,aakesson2020convolutional},
\begin{equation}
    X_j = Z_j + \theta_1 Z_{j-1} + \theta_2 Z_{j-2} + \dots + \theta_q Z_{j-q},
\end{equation}
where $j=1,\dots,p$, and $Z_j$ represents latent white noise. For the experiments in this paper, we consider $q=2$ and $Z_j \sim N(0,1)$ \cite{wiqvist2019partially}. The MA(2) model is identifiable in the following triangular region,
\begin{equation}
    \theta_1 \in [-2,2], \theta_2 \in [-1, 1], \theta_2 \pm \theta_1 \geq -1.
\end{equation}
In order to ease implementation and compatibility between the compared approaches, a rectangular uniform prior encompassing the triangular region was used in the experiments. We note here that the estimated (proposal) posterior was always within the triangular region, therefore no identifiability concerns were encountered due to a rectangular prior.

%\textcolor{red}{TODO: However we use a uniform prior...how do we motivate this choice? Both SNPE-C and ABC-SMC need a pdf of the prior. BCNN don't. Further the SBI package don't have an API supporting user-defined priors, which is baaaad}

\subsubsection{Experimental Setup}
In the experiment we conduct a total of $G=6$ iterations (rounds) of refining the (proposal) posterior estimate for 10 individual seeds. The total amount of parameter proposals in each round is set to $N=3000$ and the number of bins for BCNN is set to $B=4$ per parameter dimension, resulting in a total of 16 bins. The exploitation threshold is set to $\delta = 0.05$. We simulate the MA(2) model with 100 equidistant time points.
For the ABC-SMC method, we use auto-correlation coefficients of lag 1 and 2, and variance as summary statistics. They are non-sufficient, but are chosen as a representation of a typical choice made by a practitioner in absence of much prior knowledge. Euclidean distance was used as the discrepancy measure.

\subsubsection{Results}
Here we demonstrate the perfomance of the proposed BCNN approach and compare against SNPE-C and ABC-SMC. The model has a superficial posterior, but the map $\boldsymbol{\theta} \mapsto {\bf y}$ is ill-discriminated, i.e., considerable variation in $\boldsymbol \theta$ provides little feedback in ${\bf y}$.  %Figure~\ref{fig:grid_MA2} A shows a qualitative evaluation of two sample executions(seed) of BCNN and SNPE-C. We observed that the BCNN is more robust in estimating the posterior as compared to SNPE-C, which we further demonstrate in Figure~\ref{fig:emd_MA2} by calculating the Earth Moving Distance (EMD, also referred as Wasserstein distance) and Maximum Mean Discrapancy (MMD) between estimated posteriors and the exact posterior. Finally, we also include ABC-SMC for reference; however, this method requires a substantial increase of simulations per round for convergence.  

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_grid_4_25x4_25_fontsize8.pdf}}
\caption{MA(2) per round proposal posterior estimation for BCNN, compared to approximated posteriors obtained using SNPE-C, and ABC-SMC, together with the ground truth - a MCMC generated posterior. Each round includes $N=3000$ parameter proposals, except for ABC-SMC which require many more. BCNN and SNPE-C includes two different random seeds (1) and (2). Final column to the right shows a Gaussian kernel density estimate of the final round posterior per method.}
\label{fig:grid_MA2}
\end{center}
\vskip 0.2in
\end{figure}

Figure~\ref{fig:grid_MA2} provides qualitative insights on the shapes of the estimated proposal posterior obtained using the proposed approach, and posterior distributions obtained using the other approaches. Figure~\ref{fig:emd_MA2} depicts the log mean earth mover distance (EMD)~\cite{levina2001earth} and the mean of maximum mean discrepancy (MMD) \cite{Scholkopf_MMD} between the estimated distributions and the exact posterior. We see similar convergence patterns between BCNN and SNPE-C for both the EMD and MMD over subsequent rounds and different seeds, with the BCNN having a slight edge. This is despite the BCNN targeting a proposal posterior rather than the true posterior with a correction step. It can also be observed that the BCNN is highly robust over different seeds with very little variance in performance.
Another interesting observation from Figure~\ref{fig:grid_MA2} is the improvement or refinement extracted by each approach over subsequent rounds. It can be seen that BCNN(1) and BCNN(2) are able to substantially improve the quality of the estimated distributions between rounds 2-4 and 4-6. The ability of the BCNN architecture to greatly improve over successive rounds can be attributed to the combination of uncertainty estimates from the BCNN model leveraged by the adaptive sampling algorithm to identify regions most likely to contain the true posterior.

%The lack of variability between seeds, the error bars in the figure showcase the robust properties of the BCNN method.

\begin{figure}[ht]

\includegraphics[width=\columnwidth]{img/ma2_emd_mmd_4_25x2_fontscale08.pdf}
\caption{Target posterior error estimates, log mean EMD (left) and Mean MMD (right) for BCNN, SNPE-C and ABC-SMC per round, with variability $\pm$ 1 std, recovered from ten independent seeds. The BCNN have a slight edge over SNPE-C, especially when it comes to the variability. Note that on average ABC-SMC will require more simulations to converge than the other two, see bottom row in Figure~\ref{fig:grid_MA2}.}
\label{fig:emd_MA2}

\end{figure}

\subsection{Lotka-Volterra predator-prey model}
The Lotka-Volterra predator-prey dynamics model is a popular parameter estimation benchmark problem \cite{aakesson2020convolutional, greenberg2019automatic}. The model is implemented as a stochastic Markov jump process \cite{prangle2017adapting} simulated using the stochastic simulation algorithm (SSA) \cite{gillespie1977exact}. The model consists of three events - prey reproduction, predation and predator death. The model is described by the events:% $\mathcal{X}_1 \longrightarrow 2\mathcal{X}_1$, $ \mathcal{X}_1 + \mathcal{X}_2 \longrightarrow 2\mathcal{X}_2$ and $ \mathcal{X}_2 \longrightarrow \phi$.
\begin{equation}
 \begin{aligned}
     \mathcal{X}_1 &\xrightarrow{\theta_1} 2\mathcal{X}_1,\\
     \mathcal{X}_1 + \mathcal{X}_2 &\xrightarrow{\theta_2} 2\mathcal{X}_2,\\
     \mathcal{X}_2 &\xrightarrow{\theta_3} \emptyset.
 \end{aligned}
\end{equation}
The parameters ${\boldsymbol{\theta}} = \{ \theta_1, \theta_2, \theta_3\}$ control the events described above, and have corresponding rates $\theta_1 \mathcal{X}_1, \theta_2 \mathcal{X}_1 \mathcal{X}_2, \theta_3 \mathcal{X}_2$ respectively. 
\subsubsection{Experimental Setup}
% K = 5
% G = 8
% N = 1000
% \delta = 0.05
We study the inference using a total of $G=8$ rounds of refining the target distribution, with five repetitions using different random number seeds, setting the total amount of parameter proposals in each round to  $N=1000$. Further, we define the number of bins to $B=5$ per dimension and the threshold to be $\delta = 0.05$.
The LV model data consists of 51 observations in the time span $[0 \text{ h},50 \text{ h}]$. We construct the model and the corresponding simulator using \textit{GillesPy2}\footnote{https://github.com/StochSS/GillesPy2} \cite{gillespy_2017}. 
Finally, the initial conditions include $\mathcal{X}_1=50$ and $\mathcal{X}_2=100$, and the actual parameters to be inferred are $[\log(1.0), \log(0.005), \log(1.0)]$. The prior we apply per dimension is $\mathbb{U}(\log(0.002), \log(2))$. We include the ABC-SMC estimate, because of the lack of a true posterior solution, and the LV model is a common benchmark problem in the ABC literature, i.e., the solution it produces is well accepted \cite{SNPE_A_Papamakarios,owen2015scalable}.
For the ABC-SMC method, we use no summary statistics, i.e., the data itself is directly compared, and a Euclidean distance as the discrepancy measure. The tolerance thresholds ($\epsilon$'s) for SMC-ABC are selected using a relative scheme, where the $20$-th percentile value of the ABC distances from the previous round is selected as $\epsilon$.
\subsubsection{Results}
% MSE
Compared to the MA(2) model, we do not have an analytical posterior to compare to, thus we cannot refer to the EMD of the method. However, we solve the problem with known truth (the parameters to infer) and consider the mean square error (MSE) for computed target posteriors. The reasoning is that MSE holds information both about the bias and the variance of the distribution. But with the known limitation of not considering any higher moments. In Figure~\ref{fig:MSE_LV}, we evaluate the said MSE of the BCNN approach together with SNPE-C and ABC-SMC. The BCNN performs on par with SNPE-C, without any loss of posterior concentration; see post round 5 how the standard deviation increases per generation of SNPE-C. Exploring what might have caused the increase in standard deviation, we refer to Figure~\ref{fig:grid_LV}, which presents snapshots of a sequence of rounds from one run with the different methods. SNPE-C encounters an instability after round 4. From the MSE, we note that SNPE-C is mostly, not always, stable compared to the empirically robust BCNN.
Finally, we recognize that the initially poor performance of the ABC-SMC method can be attributed to the lack of adequately tuned summary statistics.

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_mse}}
\caption{The lines with error bars show the mean MSE with variability $\pm1$ std, recovered from five seeds with eight rounds for SNPE-C, BCNN, and ABC-SMC. All methods converge to similar MSE at the final round, with low variability. Notable is the initial increase in MSE for ABC-SMC; from Figure~\ref{fig:grid_LV} it can be inferred that the approximated posterior (proposal posterior in case of BCNN) is bi-modal before finding a suitable discrepancy threshold.}
\label{fig:MSE_LV}
\end{center}
\vskip 0.2in
\end{figure}

\begin{figure}[ht]
\vskip 0.2in
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_dens}}
\caption{Snapshots of the proposal posterior estimate for the LV problem using BCNN, and posterior estimates using SNPE-C, and ABC-SMC. Each column is a different round: 2, 4, and the final 8, and each row gives the three parameters combinations possible of $\theta_1$, $\theta_2$, and $\theta_3$. The true parameter value is indicated by the orange cross. The prior covers the entire box uniformly. For the ABC-SMC the cumulative number of trials at round 2, 4 and 8 are $[8000, 19000, 127000]$, compared to $[2000, 4000, 8000]$ for BCNN and SNPE-C.}
\label{fig:grid_LV}
\end{center}
\vskip 0.2in
\end{figure}


%\subsection{Example 3?}
% placeholder
\section{Discussion}
\label{sec:discussion}
% Gotchas and pitfalls
%\textcolor{red}{PS: Can we mention some pitfalls, something to be careful about and pointers on how to get the best out of our approach?}
% - by using classification we cannot re-use the trained network, we have to start anew each round. Meaning that the information gained from each trained network must be leveraged to it's fullest when constructing the proposal distribution.
% - number of parameter dimensions

The results presented in the previous section demonstrate that the proposed approach is comparable to SNPE-C, which is a well-established method for approximating flexible posterior densities using NNs. Notably, the comparable performance has been achieved in spite of a lack of correction of the proposal posterior in the case of the BNN. The uncertainty estimates and a simplification strategy of accomplishing regression via classification using the BNN allows the adaptive sampling algorithm to consistently drive the search for the true posterior in the right direction. This can be seen in the very low standard deviations presented in Figures~\ref{fig:emd_MA2} and \ref{fig:MSE_LV}. 
However, an important observation is that SNPE-C does not utilize any exploitation similar to $\delta$ in the proposed approach  (see Section~\ref{sec:method}). Effectively, the deviations in performance are reduced using this exploitation. 

However, incorporating a similar exploitation strategy used in SNPE methods may not alone resolve the issue of occasional, substantially inaccurate posterior estimates, such as SNPE-C(2) in Figure \ref{fig:grid_MA2}. The stability and robustness of the proposed approach stems from the Bayesian setting considered in this work. Further, the BNN is specifically tailored for models with complex data distributions, such as discrete stochastic and dynamical models. 
%The BNN functions as both  an embedding network to extract summary statistics and as a proposal posterior estimator. 
% However, the use of categorical distributions as proposal posterior estimators is limited to fairly simplistic posteriors. For flexible posterior estimators we recommend SNPE-C, but to utilize the robustness and the exploitative properties of the proposed approach one might consider using both approaches in conjunction with each other. For example, using the BNN as a pre-processing step to robustly define a prior distribution, and subsequently passing the proposal posterior estimate $\tilde{p}(\boldsymbol{\theta}'|y_0)$ as the prior for SNPE-C. This approach is outlined in Algorithm~\ref{alg:proposed}.

% \begin{algorithm}[]
%   \caption{Proposed: BNN in conjuction with SNPE-C}
%   \label{alg:proposed}
% \begin{algorithmic}
%   \STATE Do Algorithm~\ref{alg:posterior} to get $\tilde{p}_G(\boldsymbol{\theta})$
%   \STATE Do SNPE-C with $\tilde{p}_G(\boldsymbol{\theta})$ as prior
% \end{algorithmic}
% \end{algorithm}




% Significance of uncertainty estimates provided by the B(C)NN
%The results presented in the previous section underscore the importance of accounting for embedding model uncertainty towards the robustness of resulting LFI quality. The uncertainty estimates provided by the BNN allow the adaptive sampling algorithm to consistently drive the search for the true posterior in the right direction. This can be seen in the very low standard deviation of parameter inference error shown in Figures~\ref{fig:emd_MA2} and \ref{fig:MSE_LV}.

% Significance of adaptive sampling and categorical classification
%Further, adaptive sampling also enables consistent and stable refinement of the estimated posterior over successive rounds of algorithm execution. \textcolor{red}{PS: @Fredrik: pls put in some text here connecting how adaptive sampling and categorical classification is responsible for the refinement. Further, would be good to contrast with SNPE and talk about our advantages}

% Dynamical models
%\textcolor{red}{PS: why is our approach well-suited for dynamical models}
% + find "optimal" summaries, which is a hard task for time-series/dynamical models. Able to handle high dimension data well. 
\subsection{Future work}
% Future work
%\textcolor{red}{PS: Future work, explore more sophisticated sampling algorithms and the interplay between exploration and exploitation}

The convergence plots in Figure~\ref{fig:emd_MA2} and \ref{fig:MSE_LV} point towards an opportunity  for early stopping when the proposal posterior estimate no longer receives significant updates. A possibility could be to measure the highest density region (HDR)~\cite{HDR}, or EMD, change between proposal posterior estimates, and stop if the difference is below a threshold value. Using this could substantially reduce the number of simulations involved and remove the requirement of specifying the amount of rounds $G$.

Choosing the optimal number of bins $B$ and number of proposals $N$ in each bin (support) is currently not trivial. A sufficient support for dynamical models is highly dependent on the response of simulation outputs to perturbations in the parameter space. For the MA(2) model, a larger $N$ per round was required, as well as fewer bins compared to the settings for the LV problem. This was owing to the MA(2) model mapping being ill-discriminated in nature as described earlier. Thus, for the BNN classifier to sufficiently learn discriminating features, one would require more data. Further, currently the bin width is fixed and does not consider the distribution of proposals. Exploring variable bin widths based on the rate of change of proposal distributions over successive rounds could potentially improve convergence speed, and also balance the support for bins more evenly. As a remark on the choice of number of bins, in \ref{sec:hyperMA2} and \ref{sec:hyperLV} we present the result from our empirical tests, which encourages further studies.

The procedure of assigning a continuous $\theta_i$ into a bin does not take discretization uncertainty into account (e.g., when $\theta$ is close to the egde of two bins), which can lead to inaccurate training. The aforementioned problems will be considered as future work and will involve adaptive bin selection based on the discriminative ability of the BNN. Further, bin uncertainties could potentially be tackled using fuzzy set theory and other bin strategies used in regression via classification.

The hyperparameter $\delta$ (see Section~\ref{sec:method}) that controls exploitation is currently considered to be fixed. However, an adaptive $\delta$ dependent on the current round can potentially improve convergence, e.g., an exponentially decaying $\delta$. The similarity to the threshold one use in ABC-SMC, aligns the adaptivity of $\delta$ well with strategies already in practice. In \ref{sec:hyperMA2} and \ref{sec:hyperLV} we give additional test results on the subject. 

A regression-based approach as opposed to the classification based approach presented in this paper will be explored. We will also work towards integrating the proposal correction process as part of the algorithm in order to estimate the true posterior.

 %\textcolor{red}{FW: please elaborte, I don't understand}
 %One could potentially consider making it an adaptive procedure, dependent on how well the BNN resolves the previous step; if one observes a low effective sample size, then decrease the number of bins and vice versa. Additionally, currently, the bins are of equal width per dimension; by leveraging information from the proposal distribution, one could explore more informative bin widths, potentially bridging learning between rounds.


%discussion about early stopping, constructing bins, number bins and suggested exploitation. (the number of weights in BNN increase as number of bins increase, fuzzy bins?



\section{Conclusion}
\label{sec:conclusion}
This paper presented a novel Bayesian embedded neural network based approach for learning summary statistics for parameter inference of dynamical simulation models. The proposed approach leverages Bayesian neural networks for learning high-quality summary statistics as the estimated proposal posterior while also incorporating epistemic uncertainties. An adaptive sampling algorithm utilizes the estimated proposal posterior and uncertainty estimates from the Bayesian neural network to iteratively shrink the prior search space for the actual posterior distribution. An efficient classification-based density estimation approach further approximates the proposal posterior. The proposal posteriors obtained using the proposed approach are compared to approximated posteriors obtained using a well-established sequential neural posterior estimation method and the popular approximation Bayesian computation sequential Monte Carlo likelihood-free inference methods. Experiments performed on benchmark test problems show that the proposed approach is consistently more robust under various situations compared to existing methods while delivering comparable parameter inference quality.

\bibliography{references}
\bibliographystyle{icml2021}

\begin{appendix}
\onecolumn
%\icmltitle{Adaptive sampling-driven Bayesian neural networks for likelihood-free parameter inference}
\icmltitle{Robust and integrative Bayesian neural networks\\ for likelihood-free parameter inference\\
Supplementary Material}

\section{Extended Experiments for the MA(2) Test Problem:}
We investigate the effect of varying random seeds, and a comparison of execution times for the proposed approach against SNPE-C. We also explore the effect of varying the hyperparameters involved in the proposed BCNN-based approach.
\subsection{Qualitative evaluation with additional seeds}

\begin{figure*}[ht]
\begin{center}
\centerline{\includegraphics[width=\textwidth]{img/ma2_suppl_grid.pdf}}
\caption{Ten individual seeds for estimating posterior using SNPE-C (A) and proposal posteriors using BCNN (B). }
\label{fig:ma2_grid_suppl}
\end{center}
\end{figure*}


\twocolumn
In order to investigate the robustness of the proposed approach, ten distinct random seeds were used in the parameter inference task for the BNN (BCNN) and SNPE-C based methods. Figure~\ref{fig:ma2_grid_suppl} depicts the results with each row representing a seed, and each column representing a round in the sequential parameter inference process. The random walk Metropolis-Hastings MCMC method was used to calculate the exact posterior for the purpose of comparisons with BCNN and SPNE-C. 

It can be observed that the proposed BCNN-based approach is consistent across different seeds, while SNPE-C shows susceptibility in three cases (seeds 4, 5 and 9). It can also be seen that SNPE-C typically starts with a tighter estimated posterior in the first round, but relative gains over subsequent rounds are small. In comparison, BCNN is slower in the initial stages, with larger estimated proposal posteriors in the first round. However, relative gains over successive rounds are more significant with substantial refinement being achieved. This can be attributed to the exploitation offered by the binning-based classification approach involved in parameter inference using BCNN. The final distribution in the sixth round is more concentrated for the BCNN as compared to SNPE-C. 

\subsection{Embedding Network architectures}
\label{sec:arch_ma2}
The BCNN was implemented using the TensorFlow library (Keras and tensorflow probability), while the SBI package (SNPE-C) makes use of PyTorch. The architectures are shown in Table \ref{tab:ma2}.
\begin{table}[ht]
\caption{Architecture for the BCNN (top) and SNPE-C (bottom); NDE refers to neural density estimator.}
\label{tab:ma2}
%\vskip 0.15in
\begin{center}
%\begin{small}
%\begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 100, 1)]&0\\
Conv1DFlipout (ReLU) & (None, 100, 6)  &66\\
MaxPooling1D    & (None, 10, 6)&0\\
Flatten    & (None, 60)&0\\
DenseFlipout (Softmax)     &(None, 16)&1936\\
\midrule
Total weights &&2002

\end{tabular}
%\end{sc}
%\end{small}
%\end{center}

%\caption{Architecture SNPE-C, icluding embedded summary statistics network and neural density estimator (NDE) with 10 hidden units and 2 transforms}
% \label{sample-table}
% \vskip 0.15in
% \begin{center}
% \begin{small}
% \begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 100, 1)]&0\\
Conv1d (ReLU) &(None, 100, 6)  &36\\
MaxPooling1d    &(None, 10, 6)&0\\
view (Flatten)    & (None, 60)&0\\
Linear (ReLU)     &(None, 16)&488\\
NDE (MAF) &10 hidden, 2 transform&970\\
\midrule
Total weights &&1494

\end{tabular}
%\end{sc}
%\end{small}
\end{center}
%\vskip -0.1in
\end{table}

\subsection{Hyperparameters of the BNN}
\label{sec:hyperMA2}
Here we experimented with different hyperparameters for the BNN (BCNN), including different number of bins (classes) in Figure~\ref{fig:ma2_bins}, and different exploitation strategies using the threshold $\delta$ in Figure~\ref{fig:ma2_thresh}. For MA(2), four bins per parameter (16 classes) is the best performing setting, however using five bins (25 classes) results in similar performance. For the exploitation threshold $\delta$, we observe the best overall performance using $\delta=0.05$ when observing log mean EMD. For mean MMD both show similar results. Additionally, a proposed strategy of using an exponentially decaying threshold, i.e., reducing the amount of exploitation per round, performs on a level in between $\delta = 0.05$ and $\delta = 0.0$. 
\begin{figure}[ht]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_bins__4_25x2_fontscale08.pdf}}
\caption{Log mean EMD and mean MMD for BCNN with different number of classes corresponding to the number of bins.}
\label{fig:ma2_bins}
\end{center}
\end{figure}

\begin{figure}[ht]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_thresh__4_25x2_fontscale08.pdf}}
\caption{Log mean EMD and mean MMD for BCNN for different threshold $\delta$. $\delta = 0.0$ refer to no threshold being used (i.e no exploitation). $\delta = 0.05$ is the strategy used in the main article. $\delta = e$ is an exponentially decaying threshold.}
\label{fig:ma2_thresh}
\end{center}
\end{figure}

\subsection{Execution time}
Figure~\ref{fig:ma2_time} shows the time difference for the inference task between the two implementations used in this study. Note that the BCNN used for MA(2) is one joint model (2D bins), however for the LV model several co-variate BCNN models are trained.

It is important to note that differences in software implementations (TensorFlow versus PyTorch) and approaches to parallelization make direct empirical comparisons challenging. Further, factors such as the early stopping criterion will also have an effect on the result.

%Note that this is by no means a final evaluation of the computational complexity between BCNN and SNPE-C. Both have different implementations and are using various parallel and neural network packages, further factors such as early stopping criterion will also have an effect on the result. However, Figure~\ref{fig:ma2_time} shows the time difference for the inference task between the two implementations used in this study. Observe that the BCNN used for MA(2) is one joint model (2D bins), however for the LV model several co-variate BCNN models are trained. 

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_time__4_25x2_fontscale08.pdf}}
\caption{(left) Time (minutes) per round between BCNN and SNPE-C using MAF. (right) Cumulative time (minutes). Both include simulation time per round (3000 simulations/round).  }
\label{fig:ma2_time}
\end{center}
\end{figure}

%\subsection{Various neural density estimators in SNPE-C}
%\begin{figure}[ht]
%\vskip 0.2in
%\begin{center}
%\centerline{\includegraphics[width=\columnwidth]{img/ma2_suppl_NDE__4_25x2_fontscale08.pdf}}
%\caption{Text}
%\label{fig:ma2_time}
%\end{center}
%\vskip 0.2in
%\end{figure}




\section{Extended Experiments for the Lotka-Volterra (LV) Test Problem:}
The structure of extended experiments for the LV test problem follows the MA(2) setup in the text above. 
%We investigate the effect of varying random seeds, a comparison of execution times for the proposed approach against SNPE-C. We also explore the effect of varying the hyperparameters involved in the proposed BCNN-based approach.

\subsection{Qualitative evaluation with additional seeds}
%To investigate the robustness in the LFI-methods, we repeat the experiments multiple times, with five different seeds. 
The robustness of the three approaches - BCNN, SNPE-C and ABC-SMC against five different seeds is now investigated for the Lotka-Volterra predator-prey model.
In Figure~\ref{fig:lv_grid}, we show the snapshots and the final two-dimensional marginal distributions for the five independent runs. Comparison of the three approaches shows that the BCNN approach is relatively robust, while the ABC-SMC method converges slowly. It can also be seen that SNPE-C often converges to a different distribution shape as compared to BCNN and ABC-SMC. 



% \begin{figure}[ht]
% \vskip 0.2in
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_0.pdf}}
% \caption{Snapshot \# 1 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_0}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_1.pdf}}
% \caption{Snapshot \# 2 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_1}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_2.pdf}}
% \caption{Snapshot \# 3 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_2}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_3.pdf}}
% \caption{Snapshot \# 4 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_3}
% \end{center}
% \end{figure}

% \begin{figure}[ht]
% \begin{center}
% \centerline{\includegraphics[width=\columnwidth]{img/lv_dens_4.pdf}}
% \caption{Snapshot \# 5 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
% \label{fig:lv_dens_4}
% \end{center}
% \vskip 0.2in
% \end{figure}

%\twocolumn
\subsection{Embedding Network architectures}
%The BCNN was implemented in TensorFlow (Keras and tensorflow probability) while the SBI package (SNPE-C) makes use of PyTorch. 
Implementations follow the description in Section \ref{sec:arch_ma2}. The architectures are shown in Table \ref{tab:lv}.
\begin{table}[ht]
\caption{Architecture BCNN (top) and SNPE-C(bottom).}
\label{tab:lv}
%\vskip 0.15in
\begin{center}
%\begin{small}
%\begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 51, 2)]&0\\
Conv1DFlipout (ReLU) &(None, 51, 20)  &260\\
MaxPooling1D    &(None, 10, 20)&0\\
Flatten    & (None, 200)&0\\
DenseFlipout (Softmax)     &(None, 25)&10025\\
\midrule
Total weights &&10285

\end{tabular}
%\end{sc}
%\end{small}
%\end{center}

%\caption{Architecture SNPE-C, icluding embedded summary statistics network and neural density estimator (NDE) with 10 hidden units and 2 transforms}
% \label{sample-table}
% \vskip 0.15in
% \begin{center}
% \begin{small}
% \begin{sc}
\begin{tabular}{lcccr}
\toprule
Layer & Ouput shape & Weights\\
\midrule
Input    &[(None, 51, 2)]&0\\
Conv1d (ReLU) &(None, 51, 20)  &140\\
MaxPooling1d    &(None, 10, 20)&0\\
view (Flatten)    & (None, 200)&0\\
Linear (ReLU)     &(None, 25)&1608\\
NDE (MAF) &10 hidden, 2 transform&912\\
\midrule
Total weights &&2660

\end{tabular}
%\end{sc}
%\end{small}
\end{center}
%\vskip -0.1in
\end{table}

\subsection{Hyperparameters of BNN}
\label{sec:hyperLV}
As in the case of the MA(2) test problem, the hyperparameters of interest to explore in the context of the BNN aproach for LV include the number of bins (classes) and the extent of exploitation controlled by the threshold $\delta$. In Figure~\ref{fig:lv_bins}, we explore three values for the bins. As the number of bins for the discretization of continuous data increases, so does the convergence speed towards a lower MSE.
%One could theorize that they all will converge to the same MSE, however, at different rates. 

In Figure~\ref{fig:lv_thresh}, we compare the MSE for 8 rounds using a threshold value of $\delta = 0.05$ and $\delta = 0$, i.e., with or without exploitation. It can be seen that incorporating exploitation via $\delta > 0$, results in a more rapid decline in MSE as compared to the case of no exploitation. This is expected, validates the purpose of exploitation in the proposed approach. 
%As in the case of the number of bins, the two cases might converge to the same MSE; however, when using exploitation $\delta > 0$, predictably, one reaches a lower MSE faster than without.

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_bins.pdf}}
\caption{MSE for BCNN with different number of classes corresponding to the square of the number of bins per parameter, on the LV test problem.}
\label{fig:lv_bins}
\end{center}
\end{figure}

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_thresh.pdf}}
\caption{MSE for BCNN when using threshold $\delta$: $0.05$ and $0.0$, on the LV test problem.}
\label{fig:lv_thresh}
\end{center}
\end{figure}

\subsection{Execution time}
Figure~\ref{fig:lv_time} presents the execution time of the BCNN and SNPE-C approaches. However, we note that several factors affect this comparison. It can be observed that the BCNN approach is slower as compared to SNPE-C owing to the overhead of training times for multiple classifiers.
%One should observe the execution time presented in Figure~\ref{fig:lv_time}  with caution. 
The number of required classifiers depends upon the number of parameter pair-combinations, e.g., three parameters requires four models, four parameters require six models, etc. The additional training time for the BCNN in case of LV can be reduced via better parallelization of the training process of the three classifiers. Please refer to Figure~\ref{fig:ma2_time} for a comparison of execution times for a single trained model.

\begin{figure}[H]
\begin{center}
\centerline{\includegraphics[width=\columnwidth]{img/lv_time.pdf}}
\caption{Time per round (minutes) and cumulative time (minutes), for BCNN and SNPE-C, on the LV method.}
\label{fig:lv_time}
\end{center}
\end{figure}


\newpage
\onecolumn
%\begin{landscape}
\begin{sidewaysfigure}[ht]
\centering
\includegraphics[width=21cm]{img/LV_grid_seeds.pdf}
\caption{Snapshot \# 1-5 of the LV model using ABC-SMC, BCNN, and SNPE-C.}
\label{fig:lv_grid}
\end{sidewaysfigure}
%\end{landscape}
\end{appendix}

\end{document}

%Neural network-based methods for learning summary statistics have delivered promising results for likelihood-free parameter inference of stochastic simulation models. The networks map simulator output to the estimated posterior mean in the parameter space. Existing approaches require density estimation as a post-processing step building upon deterministic neural networks, and do not take network prediction uncertainty into account.This work proposes a robust integrated approach that learns summary statistics using Bayesian neural networks, and directly estimates the posterior density using categorical distributions.An adaptive sampling scheme iteratively selects simulation locations in a simulation-efficient manner to refine the predictive posterior distribution of the network.This allows for more efficient and robust convergence on comparatively large prior spaces. We demonstrate our approach on benchmark examples and compare against related methods.


% This document was modified from the file originally made available by
% Pat Langley and Andrea Danyluk for ICML-2K. This version was created
% by Iain Murray in 2018, and modified by Alexandre Bouchard in
% 2019 and 2021. Previous contributors include Dan Roy, Lise Getoor and Tobias
% Scheffer, which was slightly modified from the 2010 version by
% Thorsten Joachims & Johannes Fuernkranz, slightly modified from the
% 2009 version by Kiri Wagstaff and Sam Roweis's 2008 version, which is
% slightly modified from Prasad Tadepalli's 2007 version which is a
% lightly changed version of the previous year's version by Andrew
% Moore, which was in turn edited from those of Kristian Kersting and
% Codrina Lauth. Alex Smola contributed to the algorithmic style files.

toArxiv.bbl
\begin{thebibliography}{39}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{doi: #1}\else
  \providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi

\bibitem[Abel et~al.(2017)Abel, Drawert, Hellander, and Petzold]{gillespy_2017}
Abel, J., Drawert, B., Hellander, A., and Petzold, L.~R.
\newblock {GillesPy}: {A} {Python} {Package} for {Stochastic} {Model}
  {Building} and {Simulation}.
\newblock \emph{IEEE Life Sciences Letters}, pp.\  1--1, 2017.
\newblock ISSN 2332-7685.
\newblock \doi{10.1109/LLS.2017.2652448}.
\newblock URL \url{http://ieeexplore.ieee.org/document/7843624/}.

\bibitem[Ahmad et~al.(2012)Ahmad, Halawani, and Albidewi]{AHMAD_RvC}
Ahmad, A., Halawani, S.~M., and Albidewi, I.~A.
\newblock Novel ensemble methods for regression via classification problems.
\newblock \emph{Expert Systems with Applications}, 39\penalty0 (7):\penalty0
  6396--6401, 2012.
\newblock ISSN 0957-4174.
\newblock \doi{https://doi.org/10.1016/j.eswa.2011.12.029}.
\newblock URL
  \url{https://www.sciencedirect.com/science/article/pii/S0957417411017003}.

\bibitem[{\AA}kesson et~al.(2020){\AA}kesson, Singh, Wrede, and
  Hellander]{aakesson2020convolutional}
{\AA}kesson, M., Singh, P., Wrede, F., and Hellander, A.
\newblock Convolutional neural networks as summary statistics for approximate
  bayesian computation.
\newblock \emph{arXiv preprint arXiv:2001.11760}, 2020.

\bibitem[Blundell et~al.(2015)Blundell, Cornebise, Kavukcuoglu, and
  Wierstra]{Bayesbybackprop}
Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D.
\newblock Weight uncertainty in neural network.
\newblock In \emph{Proceedings of the 32nd International Conference on Machine
  Learning}, volume~37 of \emph{Proceedings of Machine Learning Research}, pp.\
   1613--1622, Lille, France, 07--09 Jul 2015. PMLR.
\newblock URL \url{http://proceedings.mlr.press/v37/blundell15.html}.

\bibitem[Chen et~al.(2020)Chen, Zhang, Gutmann, Courville, and
  Zhu]{chen2020neural}
Chen, Y., Zhang, D., Gutmann, M., Courville, A., and Zhu, Z.
\newblock Neural approximate sufficient statistics for implicit models.
\newblock \emph{arXiv preprint arXiv:2010.10079}, 2020.

\bibitem[Christ et~al.(2018)Christ, Braun, Neuffer, and
  Kempa-Liehr]{christ2018time}
Christ, M., Braun, N., Neuffer, J., and Kempa-Liehr, A.~W.
\newblock Time series feature extraction on basis of scalable hypothesis tests
  (tsfresh--a python package).
\newblock \emph{Neurocomputing}, 307:\penalty0 72--77, 2018.

\bibitem[Cranmer et~al.(2020)Cranmer, Brehmer, and Louppe]{Cranmer_frontierSBI}
Cranmer, K., Brehmer, J., and Louppe, G.
\newblock The frontier of simulation-based inference.
\newblock \emph{Proceedings of the National Academy of Sciences}, 117\penalty0
  (48):\penalty0 30055--30062, 2020.
\newblock ISSN 0027-8424.
\newblock \doi{10.1073/pnas.1912789117}.
\newblock URL \url{https://www.pnas.org/content/117/48/30055}.

\bibitem[Del~Moral et~al.(2012)Del~Moral, Doucet, and Jasra]{del2012adaptive}
Del~Moral, P., Doucet, A., and Jasra, A.
\newblock An adaptive sequential monte carlo method for approximate bayesian
  computation.
\newblock \emph{Statistics and computing}, 22\penalty0 (5):\penalty0
  1009--1020, 2012.

\bibitem[Dinev \& Gutmann(2018)Dinev and Gutmann]{dinev2018dynamic}
Dinev, T. and Gutmann, M.~U.
\newblock Dynamic likelihood-free inference via ratio estimation (dire).
\newblock \emph{arXiv preprint arXiv:1810.09899}, 2018.

\bibitem[Durkan et~al.(2019)Durkan, Bekasov, Murray, and Papamakarios]{NSF}
Durkan, C., Bekasov, A., Murray, I., and Papamakarios, G.
\newblock Neural spline flows.
\newblock In \emph{Advances in Neural Information Processing Systems},
  volume~32, pp.\  7511--7522. Curran Associates, Inc., 2019.
\newblock URL
  \url{https://proceedings.neurips.cc/paper/2019/file/7ac71d433f282034e088473244df8c02-Paper.pdf}.

\bibitem[Durkan et~al.(2020)Durkan, Murray, and
  Papamakarios]{SNRE_b_constrative}
Durkan, C., Murray, I., and Papamakarios, G.
\newblock On contrastive learning for likelihood-free inference.
\newblock In \emph{Proceedings of the 37th International Conference on Machine
  Learning}, volume 119 of \emph{Proceedings of Machine Learning Research},
  pp.\  2771--2781. PMLR, 13--18 Jul 2020.
\newblock URL \url{http://proceedings.mlr.press/v119/durkan20a.html}.

\bibitem[Engblom et~al.(2020)Engblom, Eriksson, and
  Widgren]{engblom2020bayesian}
Engblom, S., Eriksson, R., and Widgren, S.
\newblock Bayesian epidemiological modeling over high-resolution network data.
\newblock \emph{Epidemics}, 32:\penalty0 100399, 2020.

\bibitem[Fearnhead \& Prangle(2012)Fearnhead and
  Prangle]{fearnhead2012constructing}
Fearnhead, P. and Prangle, D.
\newblock Constructing summary statistics for approximate bayesian computation:
  semi-automatic approximate bayesian computation.
\newblock \emph{Journal of the Royal Statistical Society: Series B (Statistical
  Methodology)}, 74\penalty0 (3):\penalty0 419--474, 2012.

\bibitem[Gillespie(1977)]{gillespie1977exact}
Gillespie, D.~T.
\newblock Exact stochastic simulation of coupled chemical reactions.
\newblock \emph{The journal of physical chemistry}, 81\penalty0 (25):\penalty0
  2340--2361, 1977.

\bibitem[Greenberg et~al.(2019{\natexlab{a}})Greenberg, Nonnenmacher, and
  Macke]{APT_2019}
Greenberg, D., Nonnenmacher, M., and Macke, J.
\newblock Automatic posterior transformation for likelihood-free inference.
\newblock In \emph{Proceedings of the 36th International Conference on Machine
  Learning}, volume~97 of \emph{Proceedings of Machine Learning Research}, pp.\
   2404--2414. PMLR, 09--15 Jun 2019{\natexlab{a}}.
\newblock URL \url{http://proceedings.mlr.press/v97/greenberg19a.html}.

\bibitem[Greenberg et~al.(2019{\natexlab{b}})Greenberg, Nonnenmacher, and
  Macke]{greenberg2019automatic}
Greenberg, D., Nonnenmacher, M., and Macke, J.
\newblock Automatic posterior transformation for likelihood-free inference.
\newblock In \emph{International Conference on Machine Learning}, pp.\
  2404--2414. PMLR, 2019{\natexlab{b}}.

\bibitem[Gretton et~al.(2012)Gretton, Borgwardt, Rasch, Sch{\"o}lkopf, and
  Smola]{Scholkopf_MMD}
Gretton, A., Borgwardt, K., Rasch, M., Sch{\"o}lkopf, B., and Smola, A.
\newblock A kernel two-sample test.
\newblock \emph{Journal of Machine Learning Research}, 13:\penalty0 723--773,
  March 2012.

\bibitem[Hermans et~al.(2020)Hermans, Begy, and Louppe]{SNRE_a}
Hermans, J., Begy, V., and Louppe, G.
\newblock Likelihood-free {MCMC} with amortized approximate ratio estimators.
\newblock In \emph{Proceedings of the 37th International Conference on Machine
  Learning}, volume 119 of \emph{Proceedings of Machine Learning Research},
  pp.\  4239--4248. PMLR, 13--18 Jul 2020.
\newblock URL \url{http://proceedings.mlr.press/v119/hermans20a.html}.

\bibitem[Hyndman(1996)]{HDR}
Hyndman, R.~J.
\newblock Computing and graphing highest density regions.
\newblock \emph{The American Statistician}, 50\penalty0 (2):\penalty0 120--126,
  1996.
\newblock \doi{10.1080/00031305.1996.10474359}.
\newblock URL
  \url{https://www.tandfonline.com/doi/abs/10.1080/00031305.1996.10474359}.

\bibitem[Jiang et~al.(2017)Jiang, Wu, Zheng, and Wong]{jiang2017learning}
Jiang, B., Wu, T.-y., Zheng, C., and Wong, W.~H.
\newblock Learning summary statistic for approximate bayesian computation via
  deep neural network.
\newblock \emph{Statistica Sinica}, pp.\  1595--1618, 2017.

\bibitem[Levina \& Bickel(2001)Levina and Bickel]{levina2001earth}
Levina, E. and Bickel, P.
\newblock The earth mover's distance is the mallows distance: Some insights
  from statistics.
\newblock In \emph{Proceedings Eighth IEEE International Conference on Computer
  Vision. ICCV 2001}, volume~2, pp.\  251--256. IEEE, 2001.

\bibitem[Lotka(1920)]{lotka}
Lotka, A.~J.
\newblock Undamped oscillations derived from the law of mass action.
\newblock \emph{Journal of the American Chemical Society}, 42\penalty0
  (8):\penalty0 1595--1599, 1920.
\newblock \doi{10.1021/ja01453a010}.
\newblock URL \url{https://doi.org/10.1021/ja01453a010}.

\bibitem[Lueckmann et~al.(2017)Lueckmann, Gon{\c{c}}alves, Bassetto, {\"O}cal,
  Nonnenmacher, and Macke]{SNPE_B_lueckmann2017flexible}
Lueckmann, J.-M., Gon{\c{c}}alves, P.~J., Bassetto, G., {\"O}cal, K.,
  Nonnenmacher, M., and Macke, J.~H.
\newblock Flexible statistical inference for mechanistic models of neural
  dynamics.
\newblock In \emph{Proceedings of the 31st International Conference on Neural
  Information Processing Systems}, pp.\  1289--1299, 2017.

\bibitem[Lueckmann et~al.(2019)Lueckmann, Bassetto, Karaletsos, and
  Macke]{lueckmann2019likelihood}
Lueckmann, J.-M., Bassetto, G., Karaletsos, T., and Macke, J.~H.
\newblock Likelihood-free inference with emulator networks.
\newblock In \emph{Symposium on Advances in Approximate Bayesian Inference},
  pp.\  32--53. PMLR, 2019.

\bibitem[Lueckmann et~al.(2021)Lueckmann, Boelts, Greenberg, Gon{\c{c}}alves,
  and Macke]{lueckmann2021benchmarking}
Lueckmann, J.-M., Boelts, J., Greenberg, D.~S., Gon{\c{c}}alves, P.~J., and
  Macke, J.~H.
\newblock Benchmarking simulation-based inference.
\newblock \emph{arXiv preprint arXiv:2101.04653}, 2021.

\bibitem[Marin et~al.(2012)Marin, Pudlo, Robert, and
  Ryder]{marin2012approximate}
Marin, J.-M., Pudlo, P., Robert, C.~P., and Ryder, R.~J.
\newblock Approximate bayesian computational methods.
\newblock \emph{Statistics and Computing}, 22\penalty0 (6):\penalty0
  1167--1180, 2012.

\bibitem[Owen et~al.(2015)Owen, Wilkinson, and Gillespie]{owen2015scalable}
Owen, J., Wilkinson, D.~J., and Gillespie, C.~S.
\newblock Scalable inference for markov processes with intractable likelihoods.
\newblock \emph{Statistics and Computing}, 25\penalty0 (1):\penalty0 145--156,
  2015.

\bibitem[Papamakarios \& Murray(2016)Papamakarios and
  Murray]{SNPE_A_Papamakarios}
Papamakarios, G. and Murray, I.
\newblock Fast {$\epsilon$}-free inference of simulation models with bayesian
  conditional density estimation.
\newblock In \emph{Advances in Neural Information Processing Systems},
  volume~29, pp.\  1028--1036. Curran Associates, Inc., 2016.
\newblock URL
  \url{https://proceedings.neurips.cc/paper/2016/file/6aca97005c68f1206823815f66102863-Paper.pdf}.

\bibitem[Papamakarios et~al.(2017)Papamakarios, Pavlakou, and
  Murray]{papamakarios2017masked}
Papamakarios, G., Pavlakou, T., and Murray, I.
\newblock Masked autoregressive flow for density estimation.
\newblock In \emph{Proceedings of the 31st International Conference on Neural
  Information Processing Systems}, pp.\  2335--2344, 2017.

\bibitem[Papamakarios et~al.(2019)Papamakarios, Sterratt, and
  Murray]{papamakarios2019sequential}
Papamakarios, G., Sterratt, D., and Murray, I.
\newblock Sequential neural likelihood: Fast likelihood-free inference with
  autoregressive flows.
\newblock In \emph{The 22nd International Conference on Artificial Intelligence
  and Statistics}, pp.\  837--848. PMLR, 2019.

\bibitem[Prangle et~al.(2017)]{prangle2017adapting}
Prangle, D. et~al.
\newblock Adapting the abc distance function.
\newblock \emph{Bayesian Analysis}, 12\penalty0 (1):\penalty0 289--309, 2017.

\bibitem[Sadouk(2019)]{sadouk2019cnn}
Sadouk, L.
\newblock Cnn approaches for time series classification.
\newblock In \emph{Time Series Analysis-Data, Methods, and Applications}, pp.\
  1--23. IntechOpen, 2019.

\bibitem[Sisson et~al.(2007)Sisson, Fan, and Tanaka]{sisson2007sequential}
Sisson, S.~A., Fan, Y., and Tanaka, M.~M.
\newblock Sequential monte carlo without likelihoods.
\newblock \emph{Proceedings of the National Academy of Sciences}, 104\penalty0
  (6):\penalty0 1760--1765, 2007.

\bibitem[Sisson et~al.(2018)Sisson, Fan, and Beaumont]{sisson2018handbook}
Sisson, S.~A., Fan, Y., and Beaumont, M.
\newblock \emph{Handbook of approximate Bayesian computation}.
\newblock Chapman and Hall/CRC, 2018.

\bibitem[Tejero-Cantero et~al.(2020)Tejero-Cantero, Boelts, Deistler,
  Lueckmann, Durkan, Gon{\c{c}}alves, Greenberg, and Macke]{tejero2020sbi}
Tejero-Cantero, A., Boelts, J., Deistler, M., Lueckmann, J.-M., Durkan, C.,
  Gon{\c{c}}alves, P.~J., Greenberg, D.~S., and Macke, J.~H.
\newblock Sbi: a toolkit for simulation-based inference.
\newblock \emph{Journal of Open Source Software}, 5\penalty0 (52):\penalty0
  2505, 2020.

\bibitem[Torgo \& Gama(1996)Torgo and Gama]{torgo1996regression}
Torgo, L. and Gama, J.
\newblock Regression by classification.
\newblock In \emph{Brazilian symposium on artificial intelligence}, pp.\
  51--60. Springer, 1996.

\bibitem[Weiss \& Indurkhya(1995)Weiss and Indurkhya]{weiss1995rule}
Weiss, S.~M. and Indurkhya, N.
\newblock Rule-based machine learning methods for functional prediction.
\newblock \emph{Journal of Artificial Intelligence Research}, 3:\penalty0
  383--403, 1995.

\bibitem[Wiqvist et~al.(2019)Wiqvist, Mattei, Picchini, and
  Frellsen]{wiqvist2019partially}
Wiqvist, S., Mattei, P.-A., Picchini, U., and Frellsen, J.
\newblock Partially exchangeable networks and architectures for learning
  summary statistics in approximate bayesian computation.
\newblock In \emph{International Conference on Machine Learning}, pp.\
  6798--6807. PMLR, 2019.

\bibitem[Wood(2010)]{wood2010statistical}
Wood, S.~N.
\newblock Statistical inference for noisy nonlinear ecological dynamic systems.
\newblock \emph{Nature}, 466\penalty0 (7310):\penalty0 1102--1104, 2010.

\end{thebibliography}

references.bib
@article{tejero2020sbi,
  title={Sbi: a toolkit for simulation-based inference},
  author={Tejero-Cantero, Alvaro and Boelts, Jan and Deistler, Michael and Lueckmann, Jan-Matthis and Durkan, Conor and Gon{\c{c}}alves, Pedro J and Greenberg, David S and Macke, Jakob H},
  journal={Journal of Open Source Software},
  volume={5},
  number={52},
  pages={2505},
  year={2020}
}

@article{weiss1995rule,
  title={Rule-based machine learning methods for functional prediction},
  author={Weiss, Sholom M and Indurkhya, Nitin},
  journal={Journal of Artificial Intelligence Research},
  volume={3},
  pages={383--403},
  year={1995}
}
@inproceedings{torgo1996regression,
  title={Regression by classification},
  author={Torgo, Lu{\'\i}s and Gama, Jo{\~a}o},
  booktitle={Brazilian symposium on artificial intelligence},
  pages={51--60},
  year={1996},
  organization={Springer}
}

@article{dinev2018dynamic,
  title={Dynamic likelihood-free inference via ratio estimation (DIRE)},
  author={Dinev, Traiko and Gutmann, Michael U},
  journal={arXiv preprint arXiv:1810.09899},
  year={2018}
}
@article{Scholkopf_MMD,
  title = {A Kernel Two-Sample Test },
  author = {Gretton, A. and Borgwardt, K. and Rasch, M. and Sch{\"o}lkopf, B. and Smola, A.},
  journal = {Journal of Machine Learning Research},
  volume = {13},
  pages = {723--773},
  month = mar,
  year = {2012},
  month_numeric = {3}
}
@article{AHMAD_RvC,
title = {Novel ensemble methods for regression via classification problems},
journal = {Expert Systems with Applications},
volume = {39},
number = {7},
pages = {6396-6401},
year = {2012},
issn = {0957-4174},
doi = {https://doi.org/10.1016/j.eswa.2011.12.029},
url = {https://www.sciencedirect.com/science/article/pii/S0957417411017003},
author = {Amir Ahmad and Sami M. Halawani and Ibrahim A. Albidewi},
keywords = {Ensembles, Regression, Classification, Decision trees},
}
@article{chen2020neural,
  title={Neural Approximate Sufficient Statistics for Implicit Models},
  author={Chen, Yanzhi and Zhang, Dinghuai and Gutmann, Michael and Courville, Aaron and Zhu, Zhanxing},
  journal={arXiv preprint arXiv:2010.10079},
  year={2020}
}

@article{del2012adaptive,
  title={An adaptive sequential Monte Carlo method for approximate Bayesian computation},
  author={Del Moral, Pierre and Doucet, Arnaud and Jasra, Ajay},
  journal={Statistics and computing},
  volume={22},
  number={5},
  pages={1009--1020},
  year={2012},
  publisher={Springer}
}

@article {Cranmer_frontierSBI,
	author = {Cranmer, Kyle and Brehmer, Johann and Louppe, Gilles},
	title = {The frontier of simulation-based inference},
	volume = {117},
	number = {48},
	pages = {30055--30062},
	year = {2020},
	doi = {10.1073/pnas.1912789117},
	publisher = {National Academy of Sciences},
	abstract = {Many domains of science have developed complex simulations to describe phenomena of interest. While these simulations provide high-fidelity models, they are poorly suited for inference and lead to challenging inverse problems. We review the rapidly developing field of simulation-based inference and identify the forces giving additional momentum to the field. Finally, we describe how the frontier is expanding so that a broad audience can appreciate the profound influence these developments may have on science.There are no data associated with this paper.},
	issn = {0027-8424},
	URL = {https://www.pnas.org/content/117/48/30055},
	eprint = {https://www.pnas.org/content/117/48/30055.full.pdf},
	journal = {Proceedings of the National Academy of Sciences}
}
 @InProceedings{Bayesbybackprop, title = {Weight Uncertainty in Neural Network}, author = {Charles Blundell and Julien Cornebise and Koray Kavukcuoglu and Daan Wierstra}, booktitle = {Proceedings of the 32nd International Conference on Machine Learning}, pages = {1613--1622}, year = {2015},
  volume = {37}, series = {Proceedings of Machine Learning Research}, address = {Lille, France}, month = {07--09 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v37/blundell15.pdf}, url = {http://proceedings.mlr.press/v37/blundell15.html}, abstract = {We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the marginal likelihood. We show that this principled kind of regularisation yields comparable performance to dropout on MNIST classification. We then demonstrate how the learnt uncertainty in the weights can be used to improve generalisation in non-linear regression problems, and how this weight uncertainty can be used to drive the exploration-exploitation trade-off in reinforcement learning.} }

@article{wen2018flipout,
  title={Flipout: Efficient pseudo-independent weight perturbations on mini-batches},
  author={Wen, Yeming and Vicol, Paul and Ba, Jimmy and Tran, Dustin and Grosse, Roger},
  journal={arXiv preprint arXiv:1803.04386},
  year={2018}
}

@incollection{sadouk2019cnn,
  title={CNN approaches for time series classification},
  author={Sadouk, Lamyaa},
  booktitle={Time Series Analysis-Data, Methods, and Applications},
  pages={1--23},
  year={2019},
  publisher={IntechOpen}
}

 @inproceedings{levina2001earth,
  title={The earth mover's distance is the mallows distance: Some insights from statistics},
  author={Levina, Elizaveta and Bickel, Peter},
  booktitle={Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001},
  volume={2},
  pages={251--256},
  year={2001},
  organization={IEEE}
}

 @article{mackay1992practical,
  title={A practical Bayesian framework for backpropagation networks},
  author={MacKay, David JC},
  journal={Neural computation},
  volume={4},
  number={3},
  pages={448--472},
  year={1992},
  publisher={MIT Press}
}

 @InProceedings{APT_2019, title = {Automatic Posterior Transformation for Likelihood-Free Inference}, author = {Greenberg, David and Nonnenmacher, Marcel and Macke, Jakob}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {2404--2414}, year = {2019},
  volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/greenberg19a/greenberg19a.pdf}, url = { http://proceedings.mlr.press/v97/greenberg19a.html }, abstract = {How can one perform Bayesian inference on stochastic simulators with intractable likelihoods? A recent approach is to learn the posterior from adaptively proposed simulations using neural network-based conditional density estimators. However, existing methods are limited to a narrow range of proposal distributions or require importance weighting that can limit performance in practice. Here we present automatic posterior transformation (APT), a new sequential neural posterior estimation method for simulation-based inference. APT can modify the posterior estimate using arbitrary, dynamically updated proposals, and is compatible with powerful flow-based density estimators. It is more flexible, scalable and efficient than previous simulation-based inference techniques. APT can operate directly on high-dimensional time series and image data, opening up new applications for likelihood-free inference.} }

@article{gutmann2018likelihood,
  title={Likelihood-free inference via classification},
  author={Gutmann, Michael U and Dutta, Ritabrata and Kaski, Samuel and Corander, Jukka},
  journal={Statistics and Computing},
  volume={28},
  number={2},
  pages={411--425},
  year={2018},
  publisher={Springer}
}

@inproceedings{greenberg2019automatic,
  title={Automatic posterior transformation for likelihood-free inference},
  author={Greenberg, David and Nonnenmacher, Marcel and Macke, Jakob},
  booktitle={International Conference on Machine Learning},
  pages={2404--2414},
  year={2019},
  organization={PMLR}
}
@inproceedings{NSF,
 author = {Durkan, Conor and Bekasov, Artur and Murray, Iain and Papamakarios, George},
 booktitle = {Advances in Neural Information Processing Systems},
 pages = {7511--7522},
 publisher = {Curran Associates, Inc.},
 title = {Neural Spline Flows},
 url = {https://proceedings.neurips.cc/paper/2019/file/7ac71d433f282034e088473244df8c02-Paper.pdf},
 volume = {32},
 year = {2019}
}


@article{lueckmann2021benchmarking,
  title={Benchmarking Simulation-Based Inference},
  author={Lueckmann, Jan-Matthis and Boelts, Jan and Greenberg, David S and Gon{\c{c}}alves, Pedro J and Macke, Jakob H},
  journal={arXiv preprint arXiv:2101.04653},
  year={2021}
}
@InProceedings{SNRE_b_constrative,
  title = 	 {On Contrastive Learning for Likelihood-free Inference},
  author =       {Durkan, Conor and Murray, Iain and Papamakarios, George},
  booktitle = 	 {Proceedings of the 37th International Conference on Machine Learning},
  pages = 	 {2771--2781},
  year = 	 {2020},
  volume = 	 {119},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {13--18 Jul},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v119/durkan20a/durkan20a.pdf},
  url = 	 {
http://proceedings.mlr.press/v119/durkan20a.html
},
  abstract = 	 {Likelihood-free methods perform parameter inference in stochastic simulator models where evaluating the likelihood is intractable but sampling synthetic data is possible. One class of methods for this likelihood-free problem uses a classifier to distinguish between pairs of parameter-observation samples generated using the simulator and pairs sampled from some reference distribution, which implicitly learns a density ratio proportional to the likelihood. Another popular class of methods fits a conditional distribution to the parameter posterior directly, and a particular recent variant allows for the use of flexible neural density estimators for this task. In this work, we show that both of these approaches can be unified under a general contrastive learning scheme, and clarify how they should be run and compared.}
}



@InProceedings{SNRE_a,
  title = 	 {Likelihood-free {MCMC} with Amortized Approximate Ratio Estimators},
  author =       {Hermans, Joeri and Begy, Volodimir and Louppe, Gilles},
  booktitle = 	 {Proceedings of the 37th International Conference on Machine Learning},
  pages = 	 {4239--4248},
  year = 	 {2020},
  volume = 	 {119},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {13--18 Jul},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v119/hermans20a/hermans20a.pdf},
  url = 	 {
http://proceedings.mlr.press/v119/hermans20a.html
},
  abstract = 	 {Posterior inference with an intractable likelihood is becoming an increasingly common task in scientific domains which rely on sophisticated computer simulations. Typically, these forward models do not admit tractable densities forcing practitioners to rely on approximations. This work introduces a novel approach to address the intractability of the likelihood and the marginal model. We achieve this by learning a flexible amortized estimator which approximates the likelihood-to-evidence ratio. We demonstrate that the learned ratio estimator can be embedded in \textsc{mcmc} samplers to approximate likelihood-ratios between consecutive states in the Markov chain, allowing us to draw samples from the intractable posterior. Techniques are presented to improve the numerical stability and to measure the quality of an approximation. The accuracy of our approach is demonstrated on a variety of benchmarks against well-established techniques. Scientific applications in physics show its applicability.}
}
@inproceedings{lueckmann2019likelihood,
  title={Likelihood-free inference with emulator networks},
  author={Lueckmann, Jan-Matthis and Bassetto, Giacomo and Karaletsos, Theofanis and Macke, Jakob H},
  booktitle={Symposium on Advances in Approximate Bayesian Inference},
  pages={32--53},
  year={2019},
  organization={PMLR}
}

@inproceedings{SNPE_B_lueckmann2017flexible,
  title={Flexible statistical inference for mechanistic models of neural dynamics},
  author={Lueckmann, Jan-Matthis and Gon{\c{c}}alves, Pedro J and Bassetto, Giacomo and {\"O}cal, Kaan and Nonnenmacher, Marcel and Macke, Jakob H},
  booktitle={Proceedings of the 31st International Conference on Neural Information Processing Systems},
  pages={1289--1299},
  year={2017}
}

@article{jiang2017learning,
  title={Learning summary statistic for approximate Bayesian computation via deep neural network},
  author={Jiang, Bai and Wu, Tung-yu and Zheng, Charles and Wong, Wing H},
  journal={Statistica Sinica},
  pages={1595--1618},
  year={2017},
  publisher={JSTOR}
}

@inproceedings{papamakarios2017masked,
  title={Masked autoregressive flow for density estimation},
  author={Papamakarios, George and Pavlakou, Theo and Murray, Iain},
  booktitle={Proceedings of the 31st International Conference on Neural Information Processing Systems},
  pages={2335--2344},
  year={2017}
}

@inproceedings{papamakarios2019sequential,
  title={Sequential neural likelihood: Fast likelihood-free inference with autoregressive flows},
  author={Papamakarios, George and Sterratt, David and Murray, Iain},
  booktitle={The 22nd International Conference on Artificial Intelligence and Statistics},
  pages={837--848},
  year={2019},
  organization={PMLR}
}

@inproceedings{SNPE_A_Papamakarios,
 author = {Papamakarios, George and Murray, Iain},
 booktitle = {Advances in Neural Information Processing Systems},
 pages = {1028--1036},
 publisher = {Curran Associates, Inc.},
 title = {Fast {$\epsilon$}-free Inference of Simulation Models with Bayesian Conditional Density Estimation},
 url = {https://proceedings.neurips.cc/paper/2016/file/6aca97005c68f1206823815f66102863-Paper.pdf},
 volume = {29},
 year = {2016}
}


@article{gonccalves2020training,
  title={Training deep neural density estimators to identify mechanistic models of neural dynamics},
  author={Gon{\c{c}}alves, Pedro J and Lueckmann, Jan-Matthis and Deistler, Michael and Nonnenmacher, Marcel and {\"O}cal, Kaan and Bassetto, Giacomo and Chintaluri, Chaitanya and Podlaski, William F and Haddad, Sara A and Vogels, Tim P and others},
  journal={Elife},
  volume={9},
  pages={e56261},
  year={2020},
  publisher={eLife Sciences Publications Limited}
}

@article{aakesson2020convolutional,
  title={Convolutional neural networks as summary statistics for approximate bayesian computation},
  author={{\AA}kesson, Mattias and Singh, Prashant and Wrede, Fredrik and Hellander, Andreas},
  journal={arXiv preprint arXiv:2001.11760},
  year={2020}
}

@article{sisson2007sequential,
  title={Sequential monte carlo without likelihoods},
  author={Sisson, Scott A and Fan, Yanan and Tanaka, Mark M},
  journal={Proceedings of the National Academy of Sciences},
  volume={104},
  number={6},
  pages={1760--1765},
  year={2007},
  publisher={National Acad Sciences}
}

@article{engblom2020bayesian,
  title={Bayesian epidemiological modeling over high-resolution network data},
  author={Engblom, Stefan and Eriksson, Robin and Widgren, Stefan},
  journal={Epidemics},
  volume={32},
  pages={100399},
  year={2020},
  publisher={Elsevier}
}

@inproceedings{wiqvist2019partially,
  title={Partially exchangeable networks and architectures for learning summary statistics in approximate Bayesian computation},
  author={Wiqvist, Samuel and Mattei, Pierre-Alexandre and Picchini, Umberto and Frellsen, Jes},
  booktitle={International Conference on Machine Learning},
  pages={6798--6807},
  year={2019},
  organization={PMLR}
}


@incollection{prangle2018summary,
  author      = "Prangle, Dennis",
  title       = "Innovation and Intellectual Property Rights",
  booktitle   = "Handbook of approximate Bayesian computation",
  publisher   = "CRC Press",
  year        = 2018,
  chapter     = 5,
}

@article{wood2010statistical,
  title={Statistical inference for noisy nonlinear ecological dynamic systems},
  author={Wood, Simon N},
  journal={Nature},
  volume={466},
  number={7310},
  pages={1102--1104},
  year={2010},
  publisher={Nature Publishing Group}
}

@article{kays_protein_2019,
	title = {Protein and {RNA} quantification of multiple genes in single cells},
	volume = {66},
	issn = {1940-9818},
	doi = {10.2144/btn-2018-0130},
	abstract = {Single-cell analysis overcomes the problems of cellular heterogeneity by revealing the individual differences between cells in tissue. The current tools used to profile gene expression at the single-cell level are arduous and often require specialized equipment. We have previously developed a technique to quantify protein expression levels in single living cells. Here, we combine quantification of protein expression with absolute measurement of {mRNA} amounts of the same gene in the same cell, to profile the expression of genes at the transcriptional and translational levels. We show that high heterogeneity exists at both the {mRNA} and protein levels for multiple genes, even among monoclonal cells. We demonstrate a rapid, straightforward approach to single-cell profiling of {RNA} and protein production.},
	pages = {15--21},
	number = {1},
	journaltitle = {{BioTechniques}},
	shortjournal = {{BioTechniques}},
	author = {Kays, Ibrahim and Chen, Brian Edwin},
	date = {2019},
	pmid = {30334472},
	keywords = {Animals, Cell Line, Clustered Regularly Interspaced Short Palindromic Repeats, {CRISPR}-Cas9, fluorescence imaging, Gene Editing, Gene Expression Profiling, Gene Knock-In Techniques, Genes, Reporter, genome editing, Green Fluorescent Proteins, Immunoglobulins, Mice, Microscopy, Fluorescence, {mRNA} quantification, protein quantification, Proteins, Real-Time Polymerase Chain Reaction, Ribosomal Proteins, {RNA}, Messenger, single-cell analysis, Single-Cell Analysis, Workflow},
	file = {Full Text:C\:\\Users\\Fredrik Wrede\\Zotero\\storage\\JLTB95M9\\Kays and Chen - 2019 - Protein and RNA quantification of multiple genes i.pdf:application/pdf}
}
@article {Rei_2019,
	author = {Reimeg{\r a}rd, Johan and Danielsson, Marcus and Tarbier, Marcel and Schuster, Jens and Baskaran, Sathishkumar and Panagiotou, Styliani and Dahl, Niklas and Friedl{\"a}nder, Marc R. and Gallant, Caroline J.},
	title = {Combined mRNA and protein single cell analysis in a dynamic cellular system using SPARC},
	elocation-id = {749473},
	year = {2019},
	doi = {10.1101/749473},
	publisher = {Cold Spring Harbor Laboratory},
	abstract = {Combined measurements of mRNA and protein expression in single cells enables in-depth analysis of cellular states. We present single-cell protein and RNA co-profiling (SPARC), an approach to simultaneously measure global mRNA and large sets of intracellular protein in individual cells. Using SPARC, we show that mRNA expression fails to accurately reflect protein abundance at the time of measurement in human embryonic stem cells, although the direction of changes of mRNA and protein expression are in agreement during cellular differentiation. Moreover, protein levels of transcription factors better predict their downstream effects than do the corresponding transcripts. We further show that changes of the balance between protein and mRNA expression levels can be applied to infer expression kinetic trajectories, revealing future states of individual cells. Finally, we highlight that mRNA expression may be more varied among cells than levels of the corresponding proteins. Overall, our results demonstrate that mRNA and protein measurements in single cells provide different and complementary information regarding cell states. Accordingly, SPARC can offer valuable insights in gene expression programs of single cells.},
	URL = {https://www.biorxiv.org/content/early/2019/09/13/749473},
	eprint = {https://www.biorxiv.org/content/early/2019/09/13/749473.full.pdf},
	journal = {bioRxiv}
}


@article{lin_ultra-sensitive_2019,
	title = {Ultra-sensitive digital quantification of proteins and {mRNA} in single cells},
	volume = {10},
	rights = {2019 The Author(s)},
	issn = {2041-1723},
	url = {https://www.nature.com/articles/s41467-019-11531-z},
	doi = {10.1038/s41467-019-11531-z},
	abstract = {Simultaneous measurement of proteins and {mRNA} in single cells enables quantitative understanding and modeling of cellular functions. Here, we present an automated microfluidic system for multi-parameter and ultra-sensitive protein/{mRNA} measurements in single cells. Our technology improves the sensitivity of digital proximity ligation assay by up to 55-fold, with a detection limit of 2277 proteins per cell and with detection efficiency of as few as 29 protein molecules. Our measurements using this system reveal higher {mRNA}/protein correlation in single mammalian cells than previous estimates. Furthermore, time-lapse imaging of herpesÂ simplex virusÂ 1 infected epithelial cells enabled by our device shows that expression of {ICP}4 -a major transcription factor regulating hundreds of viral genes- is only partially correlated with viral protein counts, suggesting that many cells go through abortive infection. These results highlight the importance of high-sensitivity protein/{mRNA} quantification for understanding fundamental molecular mechanisms in individual cells.},
	pages = {3544},
	number = {1},
	journaltitle = {Nature Communications},
	author = {Lin, Jing and Jordi, Christian and Son, Minjun and Van Phan, Hoang and Drayman, Nir and Abasiyanik, Mustafa Fatih and Vistain, Luke and Tu, Hsiung-Lin and Tay, SavaÅŸ},
	urldate = {2020-07-01},
	date = {2019-08-07},
	langid = {english},
	note = {Number: 1
Publisher: Nature Publishing Group},
	file = {Full Text PDF:C\:\\Users\\Fredrik Wrede\\Zotero\\storage\\8E22QYD8\\Lin et al. - 2019 - Ultra-sensitive digital quantification of proteins.pdf:application/pdf;Snapshot:C\:\\Users\\Fredrik Wrede\\Zotero\\storage\\ALDPAVQA\\s41467-019-11531-z.html:text/html}
}

@article{10.2307/26384090,
 ISSN = {10170405, 19968507},
 URL = {http://www.jstor.org/stable/26384090},
 abstract = {Approximate Bayesian Computation (ABC) methods are used to approximate posterior distributions in models with unknown or computationally intractable likelihoods. Both the accuracy and computational efficiency of ABC depend on the choice of summary statistic, but outside of special cases where the optimal summary statistics are known, it is unclear which guiding principles can be used to construct effective summary statistics. In this paper we explore the possibility of automating the process of constructing summary statistics by training deep neural networks to predict the parameters from artificially generated data: the resulting summary statistics are approximately posterior means of the parameters. With minimal model-specific tuning, our method constructs summary statistics for the Ising model and the moving-average model, which match or exceed theoretically-motivated summary statistics in terms of the accuracies of the resulting posteriors.},
 author = {Bai Jiang and Tung-Yu Wu and Charles Zheng and Wing H. Wong},
 journal = {Statistica Sinica},
 number = {4},
 pages = {1595--1618},
 publisher = {Institute of Statistical Science, Academia Sinica},
 title = {LEARNING SUMMARY STATISTIC FOR APPROXIMATE BAYESIAN COMPUTATION VIA DEEP NEURAL NETWORK},
 volume = {27},
 year = {2017}
}
@article {Cranmer2020,
	author = {Cranmer, Kyle and Brehmer, Johann and Louppe, Gilles},
	title = {The frontier of simulation-based inference},
	elocation-id = {201912789},
	year = {2020},
	doi = {10.1073/pnas.1912789117},
	publisher = {National Academy of Sciences},
	issn = {0027-8424},
	URL = {https://www.pnas.org/content/early/2020/05/28/1912789117},
	eprint = {https://www.pnas.org/content/early/2020/05/28/1912789117.full.pdf},
	journal = {Proceedings of the National Academy of Sciences}
}

@article{gillespie1977exact,
  title={Exact stochastic simulation of coupled chemical reactions},
  author={Gillespie, Daniel T},
  journal={The journal of physical chemistry},
  volume={81},
  number={25},
  pages={2340--2361},
  year={1977},
  publisher={ACS Publications}
}

@article{prangle2017adapting,
  title={Adapting the ABC distance function},
  author={Prangle, Dennis and others},
  journal={Bayesian Analysis},
  volume={12},
  number={1},
  pages={289--309},
  year={2017},
  publisher={International Society for Bayesian Analysis}
}

@article{owen2015scalable,
  title={Scalable inference for Markov processes with intractable likelihoods},
  author={Owen, Jamie and Wilkinson, Darren J and Gillespie, Colin S},
  journal={Statistics and Computing},
  volume={25},
  number={1},
  pages={145--156},
  year={2015},
  publisher={Springer}
}

@article{marin2012approximate,
  title={Approximate Bayesian computational methods},
  author={Marin, Jean-Michel and Pudlo, Pierre and Robert, Christian P and Ryder, Robin J},
  journal={Statistics and Computing},
  volume={22},
  number={6},
  pages={1167--1180},
  year={2012},
  publisher={Springer}
}

@article{gardnertoggle,
  title={Construction of a genetic toggle switch in \emph{Escherichia coli}},
  author={Gardner, Thimothy S and Cantor, Charles R,  and Collins, James J},
  journal={Nature},
  volume={403},
  year={2000},
  pages={339--342},
}


@article{nunes2010optimal,
  title={On optimal selection of summary statistics for approximate Bayesian computation},
  author={Nunes, Matthew A and Balding, David J},
  journal={Statistical applications in genetics and molecular biology},
  volume={9},
  number={1},
  year={2010},
  publisher={De Gruyter}
}

@article{blum2013comparative,
  title={A comparative review of dimension reduction methods in approximate Bayesian computation},
  author={Blum, Michael GB and Nunes, Maria Antonieta and Prangle, Dennis and Sisson, Scott A and others},
  journal={Statistical Science},
  volume={28},
  number={2},
  pages={189--208},
  year={2013},
  publisher={Institute of Mathematical Statistics}
}

@inproceedings{heidrich2009hoeffding,
  title={Hoeffding and Bernstein races for selecting policies in evolutionary direct policy search},
  author={Heidrich-Meisner, Verena and Igel, Christian},
  booktitle={Proceedings of the 26th Annual International Conference on Machine Learning},
  pages={401--408},
  year={2009},
  organization={ACM}
}

@article{goh2008burstiness,
  title={Burstiness and memory in complex systems},
  author={Goh, K-I and Barab{\'a}si, A-L},
  journal={EPL (Europhysics Letters)},
  volume={81},
  number={4},
  pages={48002},
  year={2008},
  publisher={IOP Publishing}
}

@article{wrede2018smart,
  title={Smart computational exploration of stochastic gene regulatory network models using human-in-the-loop semi-supervised learning},
  author={Wrede, Fredrik and Hellander, Andreas},
  journal={bioRxiv},
  pages={490623},
  year={2018},
  publisher={Cold Spring Harbor Laboratory}
}

@article{barnes2012considerate,
  title={Considerate approaches to constructing summary statistics for ABC model selection},
  author={Barnes, Chris P and Filippi, Sarah and Stumpf, Michael PH and Thorne, Thomas},
  journal={Statistics and Computing},
  volume={22},
  number={6},
  pages={1181--1197},
  year={2012},
  publisher={Springer}
}

@inproceedings{bubeck2013multiple,
  title={Multiple identifications in multi-armed bandits},
  author={Bubeck, S{\'e}ebastian and Wang, Tengyao and Viswanathan, Nitin},
  booktitle={International Conference on Machine Learning},
  pages={258--265},
  year={2013}
}

@article{chen2008efficient,
  title={Efficient simulation budget allocation for selecting an optimal subset},
  author={Chen, Chun-Hung and He, Donghai and Fu, Michael and Lee, Loo Hay},
  journal={INFORMS Journal on Computing},
  volume={20},
  number={4},
  pages={579--595},
  year={2008},
  publisher={INFORMS}
}

@article{ratmann2007using,
  title={Using likelihood-free inference to compare evolutionary dynamics of the protein networks of H. pylori and P. falciparum},
  author={Ratmann, Oliver and J{\o}rgensen, Ole and Hinkley, Trevor and Stumpf, Michael and Richardson, Sylvia and Wiuf, Carsten},
  journal={PLoS Computational Biology},
  volume={3},
  number={11},
  pages={e230},
  year={2007},
  publisher={Public Library of Science}
}

@article{heggland2004estimating,
  title={Estimating functions in indirect inference},
  author={Heggland, Knut and Frigessi, Arnoldo},
  journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)},
  volume={66},
  number={2},
  pages={447--462},
  year={2004},
  publisher={Wiley Online Library}
}

@incollection{blum2010choosing,
  title={Choosing the summary statistics and the acceptance rate in approximate Bayesian computation},
  author={Blum, Michael GB},
  booktitle={Proceedings of COMPSTAT'2010},
  pages={47--56},
  year={2010},
  publisher={Springer}
}

@article{sedki2012contribution,
  title={Contribution to the discussion of Fearnhead and Prangle (2012). Constructing summary statistics for approximate Bayesian computation: Semi-automatic approximate Bayesian computation},
  author={Sedki, MA and Pudlo, P},
  journal={Journal of the Royal Statistical Society: Series B},
  volume={74},
  pages={466--467},
  year={2012}
}

@article{joyce2008approximately,
  title={Approximately sufficient statistics and Bayesian computation},
  author={Joyce, Paul and Marjoram, Paul},
  journal={Statistical applications in genetics and molecular biology},
  volume={7},
  number={1},
  year={2008},
  publisher={De Gruyter}
}

@inproceedings{kalyanakrishnan2010efficient,
  title={Efficient Selection of Multiple Bandit Arms: Theory and Practice.},
  author={Kalyanakrishnan, Shivaram and Stone, Peter},
  booktitle={ICML},
  volume={10},
  pages={511--518},
  year={2010}
}

@book{sisson2018handbook,
  title={Handbook of approximate Bayesian computation},
  author={Sisson, Scott A and Fan, Yanan and Beaumont, Mark},
  year={2018},
  publisher={Chapman and Hall/CRC}
}

@article{sunnaaker2013approximate,
  title={Approximate bayesian computation},
  author={Sunn{\aa}ker, Mikael and Busetto, Alberto Giovanni and Numminen, Elina and Corander, Jukka and Foll, Matthieu and Dessimoz, Christophe},
  journal={PLoS computational biology},
  volume={9},
  number={1},
  pages={e1002803},
  year={2013},
  publisher={Public Library of Science}
}

@article{nunes2015abctools,
  title={abctools: an R package for tuning approximate Bayesian computation analyses},
  author={Nunes, Matthew Alan and Prangle, Dennis},
  journal={The R Journal},
  volume={7},
  number={2},
  pages={189--205},
  year={2015}
}

@misc{simpleabc,
  author = {Morehead, Robert C},
  title = {SimpleABC},
  year = {2016},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/rcmorehead/simpleabc}}
}

@article{Fulcher2013-yw,
	Author = {Fulcher, Ben D and Little, Max A and Jones, Nick S},
	Journal = {J. R. Soc. Interface},
	Keywords = {aim2.bib},
	Number = 83,
	Pages = {20130048},
	Title = {Highly comparative time-series analysis: the empirical structure of time series and their methods},
	Volume = 10,
	Year = 2013}

@inproceedings{vermorel2005multi,
  title={Multi-armed bandit algorithms and empirical evaluation},
  author={Vermorel, Joannes and Mohri, Mehryar},
  booktitle={European conference on machine learning},
  pages={437--448},
  year={2005},
  organization={Springer}
}

@article{ashyraliyev2009systems,
  title={Systems biology: parameter estimation for biochemical models},
  author={Ashyraliyev, Maksat and Fomekong-Nanfack, Yves and Kaandorp, Jaap A and Blom, Joke G},
  journal={The FEBS journal},
  volume={276},
  number={4},
  pages={886--902},
  year={2009},
  publisher={Wiley Online Library}
}

@article{cash1979parameter,
  title={Parameter estimation in astronomy through application of the likelihood ratio},
  author={Cash, Webster},
  journal={The Astrophysical Journal},
  volume={228},
  pages={939--947},
  year={1979}
}

@book{beck1977parameter,
  title={Parameter estimation in engineering and science},
  author={Beck, James Vere and Arnold, Kenneth J},
  year={1977},
  publisher={James Beck}
}

@book{severini2000likelihood,
  title={Likelihood methods in statistics},
  author={Severini, Thomas A},
  year={2000},
  publisher={Oxford University Press}
}

@article{vilar2002mechanisms,
  title={Mechanisms of noise-resistance in genetic oscillators},
  author={Vilar, Jos{\'e} MG and Kueh, Hao Yuan and Barkai, Naama and Leibler, Stanislas},
  journal={Proceedings of the National Academy of Sciences},
  volume={99},
  number={9},
  pages={5988--5992},
  year={2002},
  publisher={National Acad Sciences}
}

@article{abel2016gillespy,
  title={GillesPy: A Python Package for Stochastic Model Building and Simulation},
  author={Abel, John H and Drawert, Brian and Hellander, Andreas and Petzold, Linda R},
  journal={IEEE life sciences letters},
  volume={2},
  number={3},
  pages={35--38},
  year={2016},
  publisher={IEEE}
}

@phdthesis{watkins1989learning,
  title={Learning from delayed rewards},
  author={Watkins, Christopher John Cornish Hellaby},
  year={1989},
  school={King's College, Cambridge}
}

@inproceedings{even2002pac,
  title={PAC bounds for multi-armed bandit and Markov decision processes},
  author={Even-Dar, Eyal and Mannor, Shie and Mansour, Yishay},
  booktitle={International Conference on Computational Learning Theory},
  pages={255--270},
  year={2002},
  organization={Springer}
}

@article{auer2002finite,
  title={Finite-time analysis of the multiarmed bandit problem},
  author={Auer, Peter and Cesa-Bianchi, Nicolo and Fischer, Paul},
  journal={Machine learning},
  volume={47},
  number={2-3},
  pages={235--256},
  year={2002},
  publisher={Springer}
}

@inproceedings{chapelle2011empirical,
  title={An empirical evaluation of thompson sampling},
  author={Chapelle, Olivier and Li, Lihong},
  booktitle={Advances in neural information processing systems},
  pages={2249--2257},
  year={2011}
}

@article{kuleshov2014algorithms,
  title={Algorithms for multi-armed bandit problems},
  author={Kuleshov, Volodymyr and Precup, Doina},
  journal={arXiv preprint arXiv:1402.6028},
  year={2014}
}

@article{zhou2015survey,
  title={A survey on contextual multi-armed bandits},
  author={Zhou, Li},
  journal={arXiv preprint arXiv:1508.03326},
  year={2015}
}

@inproceedings{park2016k2,
  title={K2-ABC: Approximate Bayesian Computation with Kernel Embeddings},
  author={Park, Mijung and Jitkrittum, Wittawat and Sejdinovic, Dino},
  booktitle={Artificial Intelligence and Statistics},
  pages={398--407},
  year={2016}
}

@article{pritchard1999population,
  title={Population growth of human Y chromosomes: a study of Y chromosome microsatellites.},
  author={Pritchard, Jonathan K and Seielstad, Mark T and Perez-Lezaun, Anna and Feldman, Marcus W},
  journal={Molecular biology and evolution},
  volume={16},
  number={12},
  pages={1791--1798},
  year={1999},
  publisher={Oxford University Press}
}

@article{beaumont2009adaptive,
  title={Adaptive approximate Bayesian computation},
  author={Beaumont, Mark A and Cornuet, Jean-Marie and Marin, Jean-Michel and Robert, Christian P},
  journal={Biometrika},
  volume={96},
  number={4},
  pages={983--990},
  year={2009},
  publisher={Oxford University Press}
}

@article{beaumont2002approximate,
  title={Approximate Bayesian computation in population genetics},
  author={Beaumont, Mark A and Zhang, Wenyang and Balding, David J},
  journal={Genetics},
  volume={162},
  number={4},
  pages={2025--2035},
  year={2002},
  publisher={Genetics Soc America}
}

@article{csillery2010approximate,
  title={Approximate Bayesian computation (ABC) in practice},
  author={Csill{\'e}ry, Katalin and Blum, Michael GB and Gaggiotti, Oscar E and Fran{\c{c}}ois, Olivier},
  journal={Trends in ecology \& evolution},
  volume={25},
  number={7},
  pages={410--418},
  year={2010},
  publisher={Elsevier}
}


@article{Lenive2016-zl,
	Abstract = {BACKGROUND: Gene expression is known to be an intrinsically
              stochastic process which can involve single-digit numbers of mRNA
              molecules in a cell at any given time. The modelling of such
              processes calls for the use of exact stochastic simulation
              methods, most notably the Gillespie algorithm. However, this
              stochasticity, also termed ``intrinsic noise'', does not account
              for all the variability between genetically identical cells
              growing in a homogeneous environment. Despite substantial
              experimental efforts, determining appropriate model parameters
              continues to be a challenge. Methods based on approximate
              Bayesian computation can be used to obtain posterior parameter
              distributions given the observed data. However, such inference
              procedures require large numbers of simulations of the model and
              exact stochastic simulation is computationally costly. In this
              work we focus on the specific case of trying to infer model
              parameters describing reaction rates and extrinsic noise on the
              basis of measurements of molecule numbers in individual cells at
              a given time point. RESULTS: To make the problem computationally
              tractable we develop an exact, model-specific, stochastic
              simulation algorithm for the commonly used two-state model of
              gene expression. This algorithm relies on certain assumptions and
              favourable properties of the model to forgo the simulation of the
              whole temporal trajectory of protein numbers in the system,
              instead returning only the number of protein and mRNA molecules
              present in the system at a specified time point. The
              computational gain is proportional to the number of protein
              molecules created in the system and becomes significant for
              systems involving hundreds or thousands of protein molecules.
              CONCLUSIONS: We employ this simulation algorithm with approximate
              Bayesian computation to jointly infer the model's rate and noise
              parameters from published gene expression data. Our analysis
              indicates that for most genes the extrinsic contributions to
              noise will be small to moderate but certainly are non-negligible.},
	Author = {Lenive, Oleg and W Kirk, Paul D and H Stumpf, Michael P},
	Journal = {BMC Syst. Biol.},
	Keywords = {Approximate Bayesian computation; Extrinsic noise; Gene expression; Stochastic simulation},
	Language = {en},
	Month = aug,
	Number = 1,
	Pages = {81},
	Title = {Inferring extrinsic noise from single-cell gene expression data using approximate Bayesian computation},
	Volume = 10,
	Year = 2016}


@article{Lillacci2013-yn,
	Abstract = {MOTIVATION: In the noisy cellular environment, stochastic
              fluctuations at the molecular level manifest as cell-cell
              variability at the population level that is quantifiable using
              high-throughput single-cell measurements. Such variability is
              rich with information about the cell's underlying gene regulatory
              networks, their architecture and the parameters of the
              biochemical reactions at their core. RESULTS: We report a novel
              method, called Inference for Networks of Stochastic Interactions
              among Genes using High-Throughput data (INSIGHT), for
              systematically combining high-throughput time-course flow
              cytometry measurements with computer-generated stochastic
              simulations of candidate gene network models to infer the
              network's stochastic model and all its parameters. By exploiting
              the mathematical relationships between experimental and simulated
              population histograms, INSIGHT achieves scalability, efficiency
              and accuracy while entirely avoiding approximate stochastic
              methods. We demonstrate our method on a synthetic gene network in
              bacteria and show that a detailed mechanistic model of this
              network can be estimated with high accuracy and high efficiency.
              Our method is completely general and can be used to infer models
              of signal-activated gene networks in any organism based solely on
              flow cytometry data and stochastic simulations. AVAILABILITY: A
              free C source code implementing the INSIGHT algorithm, together
              with test data is available from the authors. CONTACT:
              mustafa.khammash@bsse.ethz.ch SUPPLEMENTARY INFORMATION:
              Supplementary data are available at Bioinformatics online.},
	Author = {Lillacci, Gabriele and Khammash, Mustafa},
	Journal = {Bioinformatics},
	Language = {en},
	Month = sep,
	Number = 18,
	Pages = {2311--2319},
	Title = {The signal within the noise: efficient inference of stochastic gene regulation models using fluorescence histograms and stochastic simulations},
	Volume = 29,
	Year = 2013}


@article{Daigle2012-da,
	Abstract = {BACKGROUND: A prerequisite for the mechanistic simulation of a
              biochemical system is detailed knowledge of its kinetic
              parameters. Despite recent experimental advances, the estimation
              of unknown parameter values from observed data is still a
              bottleneck for obtaining accurate simulation results. Many
              methods exist for parameter estimation in deterministic
              biochemical systems; methods for discrete stochastic systems are
              less well developed. Given the probabilistic nature of stochastic
              biochemical models, a natural approach is to choose parameter
              values that maximize the probability of the observed data with
              respect to the unknown parameters, a.k.a. the maximum likelihood
              parameter estimates (MLEs). MLE computation for all but the
              simplest models requires the simulation of many system
              trajectories that are consistent with experimental data. For
              models with unknown parameters, this presents a computational
              challenge, as the generation of consistent trajectories can be an
              extremely rare occurrence. RESULTS: We have developed Monte Carlo
              Expectation-Maximization with Modified Cross-Entropy Method
              (MCEM(2)): an accelerated method for calculating MLEs that
              combines advances in rare event simulation with a computationally
              efficient version of the Monte Carlo expectation-maximization
              (MCEM) algorithm. Our method requires no prior knowledge
              regarding parameter values, and it automatically provides a
              multivariate parameter uncertainty estimate. We applied the
              method to five stochastic systems of increasing complexity,
              progressing from an analytically tractable pure-birth model to a
              computationally demanding model of yeast-polarization. Our
              results demonstrate that MCEM(2) substantially accelerates MLE
              computation on all tested models when compared to a stand-alone
              version of MCEM. Additionally, we show how our method identifies
              parameter values for certain classes of models more accurately
              than two recently proposed computationally efficient methods.
              CONCLUSIONS: This work provides a novel, accelerated version of a
              likelihood-based parameter estimation method that can be readily
              applied to stochastic biochemical systems. In addition, our
              results suggest opportunities for added efficiency improvements
              that will further enhance our ability to mechanistically simulate
              biological processes.},
	Author = {Daigle, Jr, Bernie J and Roh, Min K and Petzold, Linda R and Niemi, Jarad},
	Journal = {BMC Bioinformatics},
	Language = {en},
	Month = may,
	Pages = {68},
	Title = {Accelerated maximum likelihood parameter estimation for stochastic biochemical systems},
	Volume = 13,
	Year = 2012}


@article{Gillespie1976-eq,
	Author = {Gillespie, Daniel T},
	Journal = {J. Comput. Phys.},
	Keywords = {vr.bib},
	Pages = {403--434},
	Title = {A general method for numerically simulating the stochastic time evolution of coupled chemical reacting systems},
	Volume = 22,
	Year = 1976}

@article{Elowitz2002-ho,
	Author = {Elowitz, Michael B and Levine, Arnold J and Siggia, Eric D and Swain, Peter S},
	Journal = {Science},
	Language = {en},
	Month = aug,
	Number = 5584,
	Pages = {1183--1186},
	Title = {Stochastic gene expression in a single cell},
	Volume = 297,
	Year = 2002}

@article{prangle2015summary,
  title={Summary statistics in approximate Bayesian computation},
  author={Prangle, Dennis},
  journal={arXiv preprint arXiv:1512.05633},
  year={2015}
}

@incollection{robbins1985some,
  title={Some aspects of the sequential design of experiments},
  author={Robbins, Herbert},
  booktitle={Herbert Robbins Selected Papers},
  pages={169--177},
  year={1985},
  publisher={Springer}
}

@article{christ2016distributed,
  title={Distributed and parallel time series feature extraction for industrial big data applications},
  author={Christ, Maximilian and Kempa-Liehr, Andreas W and Feindt, Michael},
  journal={arXiv preprint arXiv:1610.07717},
  year={2016}
}

@article{fearnhead2012constructing,
  title={Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation},
  author={Fearnhead, Paul and Prangle, Dennis},
  journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)},
  volume={74},
  number={3},
  pages={419--474},
  year={2012},
  publisher={Wiley Online Library}
}

@article{nunes2010optimal,
  title={On optimal selection of summary statistics for approximate Bayesian computation},
  author={Nunes, Matthew A and Balding, David J},
  journal={Statistical applications in genetics and molecular biology},
  volume={9},
  number={1},
  year={2010},
  publisher={De Gruyter}
}

@article{barnes2012considerate,
  title={Considerate approaches to constructing summary statistics for ABC model selection},
  author={Barnes, Chris P and Filippi, Sarah and Stumpf, Michael PH and Thorne, Thomas},
  journal={Statistics and Computing},
  volume={22},
  number={6},
  pages={1181--1197},
  year={2012},
  publisher={Springer}
}

@article{prangle2014semi,
  title={Semi-automatic selection of summary statistics for ABC model choice},
  author={Prangle, Dennis and Fearnhead, Paul and Cox, Murray P and Biggs, Patrick J and French, Nigel P},
  journal={Statistical applications in genetics and molecular biology},
  volume={13},
  number={1},
  pages={67--82},
  year={2014},
  publisher={De Gruyter}
}

@incollection{blum2010choosing,
  title={Choosing the summary statistics and the acceptance rate in approximate Bayesian computation},
  author={Blum, Michael GB},
  booktitle={Proceedings of COMPSTAT'2010},
  pages={47--56},
  year={2010},
  publisher={Springer}
}

@article{sedki2012contribution,
  title={Contribution to the discussion of Fearnhead and Prangle (2012). Constructing summary statistics for approximate Bayesian computation: Semi-automatic approximate Bayesian computation},
  author={Sedki, MA and Pudlo, P},
  journal={Journal of the Royal Statistical Society: Series B},
  volume={74},
  pages={466--467},
  year={2012}
}

@article{aeschbacher2012novel,
  title={A novel approach for choosing summary statistics in approximate Bayesian computation},
  author={Aeschbacher, Simon and Beaumont, Mark A and Futschik, Andreas},
  journal={Genetics},
  volume={192},
  number={3},
  pages={1027--1047},
  year={2012},
  publisher={Genetics Soc America}
}

@article{fu1997estimating,
  title={Estimating the age of the common ancestor of a sample of DNA sequences.},
  author={Fu, Yun-Xin and Li, Wen-Hsiung},
  journal={Molecular biology and evolution},
  volume={14},
  number={2},
  pages={195--199},
  year={1997}
}

@article{wegmann2009efficient,
  title={Efficient approximate Bayesian computation coupled with Markov chain Monte Carlo without likelihood},
  author={Wegmann, Daniel and Leuenberger, Christoph and Excoffier, Laurent},
  journal={Genetics},
  volume={182},
  number={4},
  pages={1207--1218},
  year={2009},
  publisher={Genetics Soc America}
}

@article{blum2013comparative,
  title={A comparative review of dimension reduction methods in approximate Bayesian computation},
  author={Blum, Michael GB and Nunes, Maria Antonieta and Prangle, Dennis and Sisson, Scott A and others},
  journal={Statistical Science},
  volume={28},
  number={2},
  pages={189--208},
  year={2013},
  publisher={Institute of Mathematical Statistics}
}

@article{joyce2008approximately,
  title={Approximately sufficient statistics and Bayesian computation},
  author={Joyce, Paul and Marjoram, Paul},
  journal={Statistical applications in genetics and molecular biology},
  volume={7},
  number={1},
  year={2008},
  publisher={De Gruyter}
}

@article{barber2015rate,
  title={The rate of convergence for approximate Bayesian computation},
  author={Barber, Stuart and Voss, Jochen and Webster, Mark and others},
  journal={Electronic Journal of Statistics},
  volume={9},
  number={1},
  pages={80--105},
  year={2015},
  publisher={The Institute of Mathematical Statistics and the Bernoulli Society}
}

@article{lintusaari2017fundamentals,
  title={Fundamentals and recent developments in approximate Bayesian computation},
  author={Lintusaari, Jarno and Gutmann, Michael U and Dutta, Ritabrata and Kaski, Samuel and Corander, Jukka},
  journal={Systematic biology},
  volume={66},
  number={1},
  pages={e66--e82},
  year={2017},
  publisher={Oxford University Press}
}

@article{cornuet2008inferring,
  title={Inferring population history with DIY ABC: a user-friendly approach to approximate Bayesian computation},
  author={Cornuet, Jean-Marie and Santos, Filipe and Beaumont, Mark A and Robert, Christian P and Marin, Jean-Michel and Balding, David J and Guillemaud, Thomas and Estoup, Arnaud},
  journal={Bioinformatics},
  volume={24},
  number={23},
  pages={2713--2719},
  year={2008},
  publisher={Oxford University Press}
}


@article{diggle1984monte,
  title={Monte Carlo methods of inference for implicit statistical models},
  author={Diggle, Peter J and Gratton, Richard J},
  journal={Journal of the Royal Statistical Society. Series B (Methodological)},
  pages={193--227},
  year={1984},
  publisher={JSTOR}
}

@article{drawert2016stochastic,
  title={Stochastic simulation service: bridging the gap between the computational expert and the biologist},
  author={Drawert, Brian and Hellander, Andreas and Bales, Ben and Banerjee, Debjani and Bellesia, Giovanni and Daigle Jr, Bernie J and Douglas, Geoffrey and Gu, Mengyuan and Gupta, Anand and Hellander, Stefan and others},
  journal={PLoS computational biology},
  volume={12},
  number={12},
  pages={e1005220},
  year={2016},
  publisher={Public Library of Science}
}



@Article{Wegmann2009,
author={Wegmann, Daniel
and Leuenberger, Christoph
and Excoffier, Laurent},
title={Efficient approximate Bayesian computation coupled with Markov chain Monte Carlo without likelihood},
journal={Genetics},
year={2009},
month={Aug},
publisher={Genetics Society of America},
volume={182},
number={4},
pages={1207-1218},
keywords={Africa},
keywords={Bayes Theorem},
keywords={Emigration and Immigration},
keywords={Humans},
keywords={Likelihood Functions},
keywords={Markov Chains},
keywords={*Models, Genetic},
keywords={Monte Carlo Method},
keywords={*Population Dynamics},
keywords={*Probability},
keywords={Statistics as Topic},
abstract={Approximate Bayesian computation (ABC) techniques permit inferences in complex demographic models, but are computationally inefficient. A Markov chain Monte Carlo (MCMC) approach has been proposed (Marjoram et al. 2003), but it suffers from computational problems and poor mixing. We propose several methodological developments to overcome the shortcomings of this MCMC approach and hence realize substantial computational advances over standard ABC. The principal idea is to relax the tolerance within MCMC to permit good mixing, but retain a good approximation to the posterior by a combination of subsampling the output and regression adjustment. We also propose to use a partial least-squares (PLS) transformation to choose informative statistics. The accuracy of our approach is examined in the case of the divergence of two populations with and without migration. In that case, our ABC-MCMC approach needs considerably lower computation time to reach the same accuracy than conventional ABC. We then apply our method to a more complex case with the estimation of divergence times and migration rates between three African populations.},
note={19506307[pmid]},
note={PMC2728860[pmcid]},
note={genetics.109.102509[PII]},
issn={1943-2631},
doi={10.1534/genetics.109.102509},
url={https://www.ncbi.nlm.nih.gov/pubmed/19506307}
}


@Inbook{Paul2008,
author={Paul, Joyce
and Paul, Marjoram},
chapter={Approximately Sufficient Statistics and Bayesian Computation},
title={sagmb},
year={2008},
month={2019},
day={05-20T11:06:11.077+02:00},
volume={7},
note={1},
issn={15446115},
doi={10.2202/1544-6115.1389},
url={https://www.degruyter.com/view/j/sagmb.2008.7.1/sagmb.2008.7.1.1389/sagmb.2008.7.1.1389.xml}
}

@ARTICLE{2010arXiv1004.1112F,
       author = {{Fearnhead}, Paul and {Prangle}, Dennis},
        title = "{Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC}",
      journal = {arXiv e-prints},
     keywords = {Statistics - Methodology, Statistics - Computation},
         year = "2010",
        month = "Apr",
          eid = {arXiv:1004.1112},
        pages = {arXiv:1004.1112},
archivePrefix = {arXiv},
       eprint = {1004.1112},
 primaryClass = {stat.ME},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2010arXiv1004.1112F},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

@article {Vilar5988,
	author = {Vilar, Jos{\'e} M. G. and Kueh, Hao Yuan and Barkai, Naama and Leibler, Stanislas},
	title = {Mechanisms of noise-resistance in genetic oscillators},
	volume = {99},
	number = {9},
	pages = {5988--5992},
	year = {2002},
	doi = {10.1073/pnas.092133899},
	publisher = {National Academy of Sciences},
	abstract = {A wide range of organisms use circadian clocks to keep internal sense of daily time and regulate their behavior accordingly. Most of these clocks use intracellular genetic networks based on positive and negative regulatory elements. The integration of these {\textquotedblleft}circuits{\textquotedblright} at the cellular level imposes strong constraints on their functioning and design. Here, we study a recently proposed model [Barkai, N. \&amp; Leibler, S. (2000) Nature (London), 403, 267{\textendash}268] that incorporates just the essential elements found experimentally. We show that this type of oscillator is driven mainly by two elements: the concentration of a repressor protein and the dynamics of an activator protein forming an inactive complex with the repressor. Thus, the clock does not need to rely on mRNA dynamics to oscillate, which makes it especially resistant to fluctuations. Oscillations can be present even when the time average of the number of mRNA molecules goes below one. Under some conditions, this oscillator is not only resistant to but, paradoxically, also enhanced by the intrinsic biochemical noise.},
	issn = {0027-8424},
	URL = {https://www.pnas.org/content/99/9/5988},
	eprint = {https://www.pnas.org/content/99/9/5988.full.pdf},
	journal = {Proceedings of the National Academy of Sciences}
}

@Article{Gillespie1977,
author={Gillespie, Daniel T.},
title={Exact stochastic simulation of coupled chemical reactions},
journal={The Journal of Physical Chemistry},
year={1977},
month={Dec},
day={01},
publisher={American Chemical Society},
volume={81},
number={25},
pages={2340-2361},
issn={0022-3654},
doi={10.1021/j100540a008},
url={https://doi.org/10.1021/j100540a008}
}

@inproceedings{2019arXiv190110230W,
  title={Partially Exchangeable Networks and Architectures for Learning Summary Statistics in Approximate Bayesian Computation},
  author={Wiqvist, Samuel and Mattei, Pierre-Alexandre and Picchini, Umberto and Frellsen, Jes},
  booktitle={International Conference on Machine Learning},
  pages={6798--6807},
  year={2019}
}

@article{goh2008burstiness,
  title={Burstiness and memory in complex systems},
  author={Goh, K-I and Barab{\'a}si, A-L},
  journal={EPL (Europhysics Letters)},
  volume={81},
  number={4},
  pages={48002},
  year={2008},
  publisher={IOP Publishing}
}

@incollection{lecun1989generalization,
  title={Generalization and network design strategies},
  author={LeCun, Yann and others},
  booktitle={Connectionism in perspective},
  volume={19},
  year={1989},
  publisher={Citeseer}
}

@article{joyce2008approximately,
  title={Approximately sufficient statistics and Bayesian computation},
  author={Joyce, Paul and Marjoram, Paul},
  journal={Statistical applications in genetics and molecular biology},
  volume={7},
  number={1},
  year={2008},
  publisher={De Gruyter}
}

@inproceedings{zhou1988computation,
  title={Computation of optical flow using a neural network},
  author={Zhou, Yi-Tong and Chellappa, Rama},
  booktitle={IEEE International Conference on Neural Networks},
  volume={1998},
  pages={71--78},
  year={1988}
}

@inproceedings{ryder2018black,
  title={Black-Box Variational Inference for Stochastic Differential Equations},
  author={Ryder, Tom and Golightly, Andrew and McGough, A Stephen and Prangle, Dennis},
  booktitle={International Conference on Machine Learning},
  pages={4423--4432},
  year={2018}
}

@article{golightly2011bayesian,
  title={Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo},
  author={Golightly, Andrew and Wilkinson, Darren J},
  journal={Interface focus},
  volume={1},
  number={6},
  pages={807--820},
  year={2011},
  publisher={The Royal Society}
}

@article{HDR,
author = { Rob J.   Hyndman },
title = {Computing and Graphing Highest Density Regions},
journal = {The American Statistician},
volume = {50},
number = {2},
pages = {120-126},
year  = {1996},
publisher = {Taylor & Francis},
doi = {10.1080/00031305.1996.10474359},

URL = {
        https://www.tandfonline.com/doi/abs/10.1080/00031305.1996.10474359
},
eprint = {
        https://www.tandfonline.com/doi/pdf/10.1080/00031305.1996.10474359
    }
}


@book{goodfellow2016deep,
  title={Deep learning},
  author={Goodfellow, Ian and Bengio, Yoshua and Courville, Aaron},
  year={2016},
  publisher={MIT press}
}

@inproceedings{krizhevsky2012imagenet,
  title={Imagenet classification with deep convolutional neural networks},
  author={Krizhevsky, Alex and Sutskever, Ilya and Hinton, Geoffrey E},
  booktitle={Advances in neural information processing systems},
  pages={1097--1105},
  year={2012}
}

@article{DBLP:journals/corr/ZaheerKRPSS17,
  author    = {Manzil Zaheer and
               Satwik Kottur and
               Siamak Ravanbakhsh and
               Barnab{\'{a}}s P{\'{o}}czos and
               Ruslan Salakhutdinov and
               Alexander J. Smola},
  title     = {Deep Sets},
  journal   = {CoRR},
  volume    = {abs/1703.06114},
  year      = {2017},
  url       = {http://arxiv.org/abs/1703.06114},
  archivePrefix = {arXiv},
  eprint    = {1703.06114},
  timestamp = {Mon, 13 Aug 2018 16:46:16 +0200},
  biburl    = {https://dblp.org/rec/bib/journals/corr/ZaheerKRPSS17},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{lotka,
author = {Lotka, Alfred J.},
title = {UNDAMPED OSCILLATIONS DERIVED FROM THE LAW OF MASS ACTION.},
journal = {Journal of the American Chemical Society},
volume = {42},
number = {8},
pages = {1595-1599},
year = {1920},
doi = {10.1021/ja01453a010},

URL = {
        https://doi.org/10.1021/ja01453a010

},
eprint = {
        https://doi.org/10.1021/ja01453a010

}

}
@article{gillespy_2017,
	title = {{GillesPy}: {A} {Python} {Package} for {Stochastic} {Model} {Building} and {Simulation}},
	issn = {2332-7685},
	shorttitle = {{GillesPy}},
	url = {http://ieeexplore.ieee.org/document/7843624/},
	doi = {10.1109/LLS.2017.2652448},
	urldate = {2017-02-22},
	journal = {IEEE Life Sciences Letters},
	author = {Abel, John and Drawert, Brian and Hellander, Andreas and Petzold, Linda R},
	year = {2017},
	pages = {1--1}
}
@misc{tensorflow2015-whitepaper,
title={ {TensorFlow}: Large-Scale Machine Learning on Heterogeneous Systems},
url={https://www.tensorflow.org/},
note={Software available from tensorflow.org},
author={
    Mart\'{\i}n~Abadi and
    Ashish~Agarwal and
    Paul~Barham and
    Eugene~Brevdo and
    Zhifeng~Chen and
    Craig~Citro and
    Greg~S.~Corrado and
    Andy~Davis and
    Jeffrey~Dean and
    Matthieu~Devin and
    Sanjay~Ghemawat and
    Ian~Goodfellow and
    Andrew~Harp and
    Geoffrey~Irving and
    Michael~Isard and
    Yangqing Jia and
    Rafal~Jozefowicz and
    Lukasz~Kaiser and
    Manjunath~Kudlur and
    Josh~Levenberg and
    Dandelion~Man\'{e} and
    Rajat~Monga and
    Sherry~Moore and
    Derek~Murray and
    Chris~Olah and
    Mike~Schuster and
    Jonathon~Shlens and
    Benoit~Steiner and
    Ilya~Sutskever and
    Kunal~Talwar and
    Paul~Tucker and
    Vincent~Vanhoucke and
    Vijay~Vasudevan and
    Fernanda~Vi\'{e}gas and
    Oriol~Vinyals and
    Pete~Warden and
    Martin~Wattenberg and
    Martin~Wicke and
    Yuan~Yu and
    Xiaoqiang~Zheng},
  year={2015},
}

@article{christ2018time,
  title={Time series feature extraction on basis of scalable hypothesis tests (tsfresh--a python package)},
  author={Christ, Maximilian and Braun, Nils and Neuffer, Julius and Kempa-Liehr, Andreas W},
  journal={Neurocomputing},
  volume={307},
  pages={72--77},
  year={2018},
  publisher={Elsevier}
}
references.bib
@article{tejero2020sbi,
  title={Sbi: a toolkit for simulation-based inference},
  author={Tejero-Cantero, Alvaro and Boelts, Jan and Deistler, Michael and Lueckmann, Jan-Matthis and Durkan, Conor and Gon{\c{c}}alves, Pedro J and Greenberg, David S and Macke, Jakob H},
  journal={Journal of Open Source Software},
  volume={5},
  number={52},
  pages={2505},
  year={2020}
}

@article{weiss1995rule,
  title={Rule-based machine learning methods for functional prediction},
  author={Weiss, Sholom M and Indurkhya, Nitin},
  journal={Journal of Artificial Intelligence Research},
  volume={3},
  pages={383--403},
  year={1995}
}
@inproceedings{torgo1996regression,
  title={Regression by classification},
  author={Torgo, Lu{\'\i}s and Gama, Jo{\~a}o},
  booktitle={Brazilian symposium on artificial intelligence},
  pages={51--60},
  year={1996},
  organization={Springer}
}

@article{dinev2018dynamic,
  title={Dynamic likelihood-free inference via ratio estimation (DIRE)},
  author={Dinev, Traiko and Gutmann, Michael U},
  journal={arXiv preprint arXiv:1810.09899},
  year={2018}
}
@article{Scholkopf_MMD,
  title = {A Kernel Two-Sample Test },
  author = {Gretton, A. and Borgwardt, K. and Rasch, M. and Sch{\"o}lkopf, B. and Smola, A.},
  journal = {Journal of Machine Learning Research},
  volume = {13},
  pages = {723--773},
  month = mar,
  year = {2012},
  month_numeric = {3}
}
@article{AHMAD_RvC,
title = {Novel ensemble methods for regression via classification problems},
journal = {Expert Systems with Applications},
volume = {39},
number = {7},
pages = {6396-6401},
year = {2012},
issn = {0957-4174},
doi = {https://doi.org/10.1016/j.eswa.2011.12.029},
url = {https://www.sciencedirect.com/science/article/pii/S0957417411017003},
author = {Amir Ahmad and Sami M. Halawani and Ibrahim A. Albidewi},
keywords = {Ensembles, Regression, Classification, Decision trees},
}
@article{chen2020neural,
  title={Neural Approximate Sufficient Statistics for Implicit Models},
  author={Chen, Yanzhi and Zhang, Dinghuai and Gutmann, Michael and Courville, Aaron and Zhu, Zhanxing},
  journal={arXiv preprint arXiv:2010.10079},
  year={2020}
}

@article{del2012adaptive,
  title={An adaptive sequential Monte Carlo method for approximate Bayesian computation},
  author={Del Moral, Pierre and Doucet, Arnaud and Jasra, Ajay},
  journal={Statistics and computing},
  volume={22},
  number={5},
  pages={1009--1020},
  year={2012},
  publisher={Springer}
}

@article {Cranmer_frontierSBI,
	author = {Cranmer, Kyle and Brehmer, Johann and Louppe, Gilles},
	title = {The frontier of simulation-based inference},
	volume = {117},
	number = {48},
	pages = {30055--30062},
	year = {2020},
	doi = {10.1073/pnas.1912789117},
	publisher = {National Academy of Sciences},
	abstract = {Many domains of science have developed complex simulations to describe phenomena of interest. While these simulations provide high-fidelity models, they are poorly suited for inference and lead to challenging inverse problems. We review the rapidly developing field of simulation-based inference and identify the forces giving additional momentum to the field. Finally, we describe how the frontier is expanding so that a broad audience can appreciate the profound influence these developments may have on science.There are no data associated with this paper.},
	issn = {0027-8424},
	URL = {https://www.pnas.org/content/117/48/30055},
	eprint = {https://www.pnas.org/content/117/48/30055.full.pdf},
	journal = {Proceedings of the National Academy of Sciences}
}
 @InProceedings{Bayesbybackprop, title = {Weight Uncertainty in Neural Network}, author = {Charles Blundell and Julien Cornebise and Koray Kavukcuoglu and Daan Wierstra}, booktitle = {Proceedings of the 32nd International Conference on Machine Learning}, pages = {1613--1622}, year = {2015},
  volume = {37}, series = {Proceedings of Machine Learning Research}, address = {Lille, France}, month = {07--09 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v37/blundell15.pdf}, url = {http://proceedings.mlr.press/v37/blundell15.html}, abstract = {We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the marginal likelihood. We show that this principled kind of regularisation yields comparable performance to dropout on MNIST classification. We then demonstrate how the learnt uncertainty in the weights can be used to improve generalisation in non-linear regression problems, and how this weight uncertainty can be used to drive the exploration-exploitation trade-off in reinforcement learning.} }

@article{wen2018flipout,
  title={Flipout: Efficient pseudo-independent weight perturbations on mini-batches},
  author={Wen, Yeming and Vicol, Paul and Ba, Jimmy and Tran, Dustin and Grosse, Roger},
  journal={arXiv preprint arXiv:1803.04386},
  year={2018}
}

@incollection{sadouk2019cnn,
  title={CNN approaches for time series classification},
  author={Sadouk, Lamyaa},
  booktitle={Time Series Analysis-Data, Methods, and Applications},
  pages={1--23},
  year={2019},
  publisher={IntechOpen}
}

 @inproceedings{levina2001earth,
  title={The earth mover's distance is the mallows distance: Some insights from statistics},
  author={Levina, Elizaveta and Bickel, Peter},
  booktitle={Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001},
  volume={2},
  pages={251--256},
  year={2001},
  organization={IEEE}
}

 @article{mackay1992practical,
  title={A practical Bayesian framework for backpropagation networks},
  author={MacKay, David JC},
  journal={Neural computation},
  volume={4},
  number={3},
  pages={448--472},
  year={1992},
  publisher={MIT Press}
}

 @InProceedings{APT_2019, title = {Automatic Posterior Transformation for Likelihood-Free Inference}, author = {Greenberg, David and Nonnenmacher, Marcel and Macke, Jakob}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {2404--2414}, year = {2019},
  volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/greenberg19a/greenberg19a.pdf}, url = { http://proceedings.mlr.press/v97/greenberg19a.html }, abstract = {How can one perform Bayesian inference on stochastic simulators with intractable likelihoods? A recent approach is to learn the posterior from adaptively proposed simulations using neural network-based conditional density estimators. However, existing methods are limited to a narrow range of proposal distributions or require importance weighting that can limit performance in practice. Here we present automatic posterior transformation (APT), a new sequential neural posterior estimation method for simulation-based inference. APT can modify the posterior estimate using arbitrary, dynamically updated proposals, and is compatible with powerful flow-based density estimators. It is more flexible, scalable and efficient than previous simulation-based inference techniques. APT can operate directly on high-dimensional time series and image data, opening up new applications for likelihood-free inference.} }

@article{gutmann2018likelihood,
  title={Likelihood-free inference via classification},
  author={Gutmann, Michael U and Dutta, Ritabrata and Kaski, Samuel and Corander, Jukka},
  journal={Statistics and Computing},
  volume={28},
  number={2},
  pages={411--425},
  year={2018},
  publisher={Springer}
}

@inproceedings{greenberg2019automatic,
  title={Automatic posterior transformation for likelihood-free inference},
  author={Greenberg, David and Nonnenmacher, Marcel and Macke, Jakob},
  booktitle={International Conference on Machine Learning},
  pages={2404--2414},
  year={2019},
  organization={PMLR}
}
@inproceedings{NSF,
 author = {Durkan, Conor and Bekasov, Artur and Murray, Iain and Papamakarios, George},
 booktitle = {Advances in Neural Information Processing Systems},
 pages = {7511--7522},
 publisher = {Curran Associates, Inc.},
 title = {Neural Spline Flows},
 url = {https://proceedings.neurips.cc/paper/2019/file/7ac71d433f282034e088473244df8c02-Paper.pdf},
 volume = {32},
 year = {2019}
}


@article{lueckmann2021benchmarking,
  title={Benchmarking Simulation-Based Inference},
  author={Lueckmann, Jan-Matthis and Boelts, Jan and Greenberg, David S and Gon{\c{c}}alves, Pedro J and Macke, Jakob H},
  journal={arXiv preprint arXiv:2101.04653},
  year={2021}
}
@InProceedings{SNRE_b_constrative,
  title = 	 {On Contrastive Learning for Likelihood-free Inference},
  author =       {Durkan, Conor and Murray, Iain and Papamakarios, George},
  booktitle = 	 {Proceedings of the 37th International Conference on Machine Learning},
  pages = 	 {2771--2781},
  year = 	 {2020},
  volume = 	 {119},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {13--18 Jul},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v119/durkan20a/durkan20a.pdf},
  url = 	 {
http://proceedings.mlr.press/v119/durkan20a.html
},
  abstract = 	 {Likelihood-free methods perform parameter inference in stochastic simulator models where evaluating the likelihood is intractable but sampling synthetic data is possible. One class of methods for this likelihood-free problem uses a classifier to distinguish between pairs of parameter-observation samples generated using the simulator and pairs sampled from some reference distribution, which implicitly learns a density ratio proportional to the likelihood. Another popular class of methods fits a conditional distribution to the parameter posterior directly, and a particular recent variant allows for the use of flexible neural density estimators for this task. In this work, we show that both of these approaches can be unified under a general contrastive learning scheme, and clarify how they should be run and compared.}
}



@InProceedings{SNRE_a,
  title = 	 {Likelihood-free {MCMC} with Amortized Approximate Ratio Estimators},
  author =       {Hermans, Joeri and Begy, Volodimir and Louppe, Gilles},
  booktitle = 	 {Proceedings of the 37th International Conference on Machine Learning},
  pages = 	 {4239--4248},
  year = 	 {2020},
  volume = 	 {119},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {13--18 Jul},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v119/hermans20a/hermans20a.pdf},
  url = 	 {
http://proceedings.mlr.press/v119/hermans20a.html
},
  abstract = 	 {Posterior inference with an intractable likelihood is becoming an increasingly common task in scientific domains which rely on sophisticated computer simulations. Typically, these forward models do not admit tractable densities forcing practitioners to rely on approximations. This work introduces a novel approach to address the intractability of the likelihood and the marginal model. We achieve this by learning a flexible amortized estimator which approximates the likelihood-to-evidence ratio. We demonstrate that the learned ratio estimator can be embedded in \textsc{mcmc} samplers to approximate likelihood-ratios between consecutive states in the Markov chain, allowing us to draw samples from the intractable posterior. Techniques are presented to improve the numerical stability and to measure the quality of an approximation. The accuracy of our approach is demonstrated on a variety of benchmarks against well-established techniques. Scientific applications in physics show its applicability.}
}
@inproceedings{lueckmann2019likelihood,
  title={Likelihood-free inference with emulator networks},
  author={Lueckmann, Jan-Matthis and Bassetto, Giacomo and Karaletsos, Theofanis and Macke, Jakob H},
  booktitle={Symposium on Advances in Approximate Bayesian Inference},
  pages={32--53},
  year={2019},
  organization={PMLR}
}

@inproceedings{SNPE_B_lueckmann2017flexible,
  title={Flexible statistical inference for mechanistic models of neural dynamics},
  author={Lueckmann, Jan-Matthis and Gon{\c{c}}alves, Pedro J and Bassetto, Giacomo and {\"O}cal, Kaan and Nonnenmacher, Marcel and Macke, Jakob H},
  booktitle={Proceedings of the 31st International Conference on Neural Information Processing Systems},
  pages={1289--1299},
  year={2017}
}

@article{jiang2017learning,
  title={Learning summary statistic for approximate Bayesian computation via deep neural network},
  author={Jiang, Bai and Wu, Tung-yu and Zheng, Charles and Wong, Wing H},
  journal={Statistica Sinica},
  pages={1595--1618},
  year={2017},
  publisher={JSTOR}
}

@inproceedings{papamakarios2017masked,
  title={Masked autoregressive flow for density estimation},
  author={Papamakarios, George and Pavlakou, Theo and Murray, Iain},
  booktitle={Proceedings of the 31st International Conference on Neural Information Processing Systems},
  pages={2335--2344},
  year={2017}
}

@inproceedings{papamakarios2019sequential,
  title={Sequential neural likelihood: Fast likelihood-free inference with autoregressive flows},
  author={Papamakarios, George and Sterratt, David and Murray, Iain},
  booktitle={The 22nd International Conference on Artificial Intelligence and Statistics},
  pages={837--848},
  year={2019},
  organization={PMLR}
}

@inproceedings{SNPE_A_Papamakarios,
 author = {Papamakarios, George and Murray, Iain},
 booktitle = {Advances in Neural Information Processing Systems},
 pages = {1028--1036},
 publisher = {Curran Associates, Inc.},
 title = {Fast {$\epsilon$}-free Inference of Simulation Models with Bayesian Conditional Density Estimation},
 url = {https://proceedings.neurips.cc/paper/2016/file/6aca97005c68f1206823815f66102863-Paper.pdf},
 volume = {29},
 year = {2016}
}


@article{gonccalves2020training,
  title={Training deep neural density estimators to identify mechanistic models of neural dynamics},
  author={Gon{\c{c}}alves, Pedro J and Lueckmann, Jan-Matthis and Deistler, Michael and Nonnenmacher, Marcel and {\"O}cal, Kaan and Bassetto, Giacomo and Chintaluri, Chaitanya and Podlaski, William F and Haddad, Sara A and Vogels, Tim P and others},
  journal={Elife},
  volume={9},
  pages={e56261},
  year={2020},
  publisher={eLife Sciences Publications Limited}
}

@article{aakesson2020convolutional,
  title={Convolutional neural networks as summary statistics for approximate bayesian computation},
  author={{\AA}kesson, Mattias and Singh, Prashant and Wrede, Fredrik and Hellander, Andreas},
  journal={arXiv preprint arXiv:2001.11760},
  year={2020}
}

@article{sisson2007sequential,
  title={Sequential monte carlo without likelihoods},
  author={Sisson, Scott A and Fan, Yanan and Tanaka, Mark M},
  journal={Proceedings of the National Academy of Sciences},
  volume={104},
  number={6},
  pages={1760--1765},
  year={2007},
  publisher={National Acad Sciences}
}

@article{engblom2020bayesian,
  title={Bayesian epidemiological modeling over high-resolution network data},
  author={Engblom, Stefan and Eriksson, Robin and Widgren, Stefan},
  journal={Epidemics},
  volume={32},
  pages={100399},
  year={2020},
  publisher={Elsevier}
}

@inproceedings{wiqvist2019partially,
  title={Partially exchangeable networks and architectures for learning summary statistics in approximate Bayesian computation},
  author={Wiqvist, Samuel and Mattei, Pierre-Alexandre and Picchini, Umberto and Frellsen, Jes},
  booktitle={International Conference on Machine Learning},
  pages={6798--6807},
  year={2019},
  organization={PMLR}
}


@incollection{prangle2018summary,
  author      = "Prangle, Dennis",
  title       = "Innovation and Intellectual Property Rights",
  booktitle   = "Handbook of approximate Bayesian computation",
  publisher   = "CRC Press",
  year        = 2018,
  chapter     = 5,
}

@article{wood2010statistical,
  title={Statistical inference for noisy nonlinear ecological dynamic systems},
  author={Wood, Simon N},
  journal={Nature},
  volume={466},
  number={7310},
  pages={1102--1104},
  year={2010},
  publisher={Nature Publishing Group}
}

@article{kays_protein_2019,
	title = {Protein and {RNA} quantification of multiple genes in single cells},
	volume = {66},
	issn = {1940-9818},
	doi = {10.2144/btn-2018-0130},
	abstract = {Single-cell analysis overcomes the problems of cellular heterogeneity by revealing the individual differences between cells in tissue. The current tools used to profile gene expression at the single-cell level are arduous and often require specialized equipment. We have previously developed a technique to quantify protein expression levels in single living cells. Here, we combine quantification of protein expression with absolute measurement of {mRNA} amounts of the same gene in the same cell, to profile the expression of genes at the transcriptional and translational levels. We show that high heterogeneity exists at both the {mRNA} and protein levels for multiple genes, even among monoclonal cells. We demonstrate a rapid, straightforward approach to single-cell profiling of {RNA} and protein production.},
	pages = {15--21},
	number = {1},
	journaltitle = {{BioTechniques}},
	shortjournal = {{BioTechniques}},
	author = {Kays, Ibrahim and Chen, Brian Edwin},
	date = {2019},
	pmid = {30334472},
	keywords = {Animals, Cell Line, Clustered Regularly Interspaced Short Palindromic Repeats, {CRISPR}-Cas9, fluorescence imaging, Gene Editing, Gene Expression Profiling, Gene Knock-In Techniques, Genes, Reporter, genome editing, Green Fluorescent Proteins, Immunoglobulins, Mice, Microscopy, Fluorescence, {mRNA} quantification, protein quantification, Proteins, Real-Time Polymerase Chain Reaction, Ribosomal Proteins, {RNA}, Messenger, single-cell analysis, Single-Cell Analysis, Workflow},
	file = {Full Text:C\:\\Users\\Fredrik Wrede\\Zotero\\storage\\JLTB95M9\\Kays and Chen - 2019 - Protein and RNA quantification of multiple genes i.pdf:application/pdf}
}
@article {Rei_2019,
	author = {Reimeg{\r a}rd, Johan and Danielsson, Marcus and Tarbier, Marcel and Schuster, Jens and Baskaran, Sathishkumar and Panagiotou, Styliani and Dahl, Niklas and Friedl{\"a}nder, Marc R. and Gallant, Caroline J.},
	title = {Combined mRNA and protein single cell analysis in a dynamic cellular system using SPARC},
	elocation-id = {749473},
	year = {2019},
	doi = {10.1101/749473},
	publisher = {Cold Spring Harbor Laboratory},
	abstract = {Combined measurements of mRNA and protein expression in single cells enables in-depth analysis of cellular states. We present single-cell protein and RNA co-profiling (SPARC), an approach to simultaneously measure global mRNA and large sets of intracellular protein in individual cells. Using SPARC, we show that mRNA expression fails to accurately reflect protein abundance at the time of measurement in human embryonic stem cells, although the direction of changes of mRNA and protein expression are in agreement during cellular differentiation. Moreover, protein levels of transcription factors better predict their downstream effects than do the corresponding transcripts. We further show that changes of the balance between protein and mRNA expression levels can be applied to infer expression kinetic trajectories, revealing future states of individual cells. Finally, we highlight that mRNA expression may be more varied among cells than levels of the corresponding proteins. Overall, our results demonstrate that mRNA and protein measurements in single cells provide different and complementary information regarding cell states. Accordingly, SPARC can offer valuable insights in gene expression programs of single cells.},
	URL = {https://www.biorxiv.org/content/early/2019/09/13/749473},
	eprint = {https://www.biorxiv.org/content/early/2019/09/13/749473.full.pdf},
	journal = {bioRxiv}
}


@article{lin_ultra-sensitive_2019,
	title = {Ultra-sensitive digital quantification of proteins and {mRNA} in single cells},
	volume = {10},
	rights = {2019 The Author(s)},
	issn = {2041-1723},
	url = {https://www.nature.com/articles/s41467-019-11531-z},
	doi = {10.1038/s41467-019-11531-z},
	abstract = {Simultaneous measurement of proteins and {mRNA} in single cells enables quantitative understanding and modeling of cellular functions. Here, we present an automated microfluidic system for multi-parameter and ultra-sensitive protein/{mRNA} measurements in single cells. Our technology improves the sensitivity of digital proximity ligation assay by up to 55-fold, with a detection limit of 2277 proteins per cell and with detection efficiency of as few as 29 protein molecules. Our measurements using this system reveal higher {mRNA}/protein correlation in single mammalian cells than previous estimates. Furthermore, time-lapse imaging of herpesÂ simplex virusÂ 1 infected epithelial cells enabled by our device shows that expression of {ICP}4 -a major transcription factor regulating hundreds of viral genes- is only partially correlated with viral protein counts, suggesting that many cells go through abortive infection. These results highlight the importance of high-sensitivity protein/{mRNA} quantification for understanding fundamental molecular mechanisms in individual cells.},
	pages = {3544},
	number = {1},
	journaltitle = {Nature Communications},
	author = {Lin, Jing and Jordi, Christian and Son, Minjun and Van Phan, Hoang and Drayman, Nir and Abasiyanik, Mustafa Fatih and Vistain, Luke and Tu, Hsiung-Lin and Tay, SavaÅŸ},
	urldate = {2020-07-01},
	date = {2019-08-07},
	langid = {english},
	note = {Number: 1
Publisher: Nature Publishing Group},
	file = {Full Text PDF:C\:\\Users\\Fredrik Wrede\\Zotero\\storage\\8E22QYD8\\Lin et al. - 2019 - Ultra-sensitive digital quantification of proteins.pdf:application/pdf;Snapshot:C\:\\Users\\Fredrik Wrede\\Zotero\\storage\\ALDPAVQA\\s41467-019-11531-z.html:text/html}
}

@article{10.2307/26384090,
 ISSN = {10170405, 19968507},
 URL = {http://www.jstor.org/stable/26384090},
 abstract = {Approximate Bayesian Computation (ABC) methods are used to approximate posterior distributions in models with unknown or computationally intractable likelihoods. Both the accuracy and computational efficiency of ABC depend on the choice of summary statistic, but outside of special cases where the optimal summary statistics are known, it is unclear which guiding principles can be used to construct effective summary statistics. In this paper we explore the possibility of automating the process of constructing summary statistics by training deep neural networks to predict the parameters from artificially generated data: the resulting summary statistics are approximately posterior means of the parameters. With minimal model-specific tuning, our method constructs summary statistics for the Ising model and the moving-average model, which match or exceed theoretically-motivated summary statistics in terms of the accuracies of the resulting posteriors.},
 author = {Bai Jiang and Tung-Yu Wu and Charles Zheng and Wing H. Wong},
 journal = {Statistica Sinica},
 number = {4},
 pages = {1595--1618},
 publisher = {Institute of Statistical Science, Academia Sinica},
 title = {LEARNING SUMMARY STATISTIC FOR APPROXIMATE BAYESIAN COMPUTATION VIA DEEP NEURAL NETWORK},
 volume = {27},
 year = {2017}
}
@article {Cranmer2020,
	author = {Cranmer, Kyle and Brehmer, Johann and Louppe, Gilles},
	title = {The frontier of simulation-based inference},
	elocation-id = {201912789},
	year = {2020},
	doi = {10.1073/pnas.1912789117},
	publisher = {National Academy of Sciences},
	issn = {0027-8424},
	URL = {https://www.pnas.org/content/early/2020/05/28/1912789117},
	eprint = {https://www.pnas.org/content/early/2020/05/28/1912789117.full.pdf},
	journal = {Proceedings of the National Academy of Sciences}
}

@article{gillespie1977exact,
  title={Exact stochastic simulation of coupled chemical reactions},
  author={Gillespie, Daniel T},
  journal={The journal of physical chemistry},
  volume={81},
  number={25},
  pages={2340--2361},
  year={1977},
  publisher={ACS Publications}
}

@article{prangle2017adapting,
  title={Adapting the ABC distance function},
  author={Prangle, Dennis and others},
  journal={Bayesian Analysis},
  volume={12},
  number={1},
  pages={289--309},
  year={2017},
  publisher={International Society for Bayesian Analysis}
}

@article{owen2015scalable,
  title={Scalable inference for Markov processes with intractable likelihoods},
  author={Owen, Jamie and Wilkinson, Darren J and Gillespie, Colin S},
  journal={Statistics and Computing},
  volume={25},
  number={1},
  pages={145--156},
  year={2015},
  publisher={Springer}
}

@article{marin2012approximate,
  title={Approximate Bayesian computational methods},
  author={Marin, Jean-Michel and Pudlo, Pierre and Robert, Christian P and Ryder, Robin J},
  journal={Statistics and Computing},
  volume={22},
  number={6},
  pages={1167--1180},
  year={2012},
  publisher={Springer}
}

@article{gardnertoggle,
  title={Construction of a genetic toggle switch in \emph{Escherichia coli}},
  author={Gardner, Thimothy S and Cantor, Charles R,  and Collins, James J},
  journal={Nature},
  volume={403},
  year={2000},
  pages={339--342},
}


@article{nunes2010optimal,
  title={On optimal selection of summary statistics for approximate Bayesian computation},
  author={Nunes, Matthew A and Balding, David J},
  journal={Statistical applications in genetics and molecular biology},
  volume={9},
  number={1},
  year={2010},
  publisher={De Gruyter}
}

@article{blum2013comparative,
  title={A comparative review of dimension reduction methods in approximate Bayesian computation},
  author={Blum, Michael GB and Nunes, Maria Antonieta and Prangle, Dennis and Sisson, Scott A and others},
  journal={Statistical Science},
  volume={28},
  number={2},
  pages={189--208},
  year={2013},
  publisher={Institute of Mathematical Statistics}
}

@inproceedings{heidrich2009hoeffding,
  title={Hoeffding and Bernstein races for selecting policies in evolutionary direct policy search},
  author={Heidrich-Meisner, Verena and Igel, Christian},
  booktitle={Proceedings of the 26th Annual International Conference on Machine Learning},
  pages={401--408},
  year={2009},
  organization={ACM}
}

@article{goh2008burstiness,
  title={Burstiness and memory in complex systems},
  author={Goh, K-I and Barab{\'a}si, A-L},
  journal={EPL (Europhysics Letters)},
  volume={81},
  number={4},
  pages={48002},
  year={2008},
  publisher={IOP Publishing}
}

@article{wrede2018smart,
  title={Smart computational exploration of stochastic gene regulatory network models using human-in-the-loop semi-supervised learning},
  author={Wrede, Fredrik and Hellander, Andreas},
  journal={bioRxiv},
  pages={490623},
  year={2018},
  publisher={Cold Spring Harbor Laboratory}
}

@article{barnes2012considerate,
  title={Considerate approaches to constructing summary statistics for ABC model selection},
  author={Barnes, Chris P and Filippi, Sarah and Stumpf, Michael PH and Thorne, Thomas},
  journal={Statistics and Computing},
  volume={22},
  number={6},
  pages={1181--1197},
  year={2012},
  publisher={Springer}
}

@inproceedings{bubeck2013multiple,
  title={Multiple identifications in multi-armed bandits},
  author={Bubeck, S{\'e}ebastian and Wang, Tengyao and Viswanathan, Nitin},
  booktitle={International Conference on Machine Learning},
  pages={258--265},
  year={2013}
}

@article{chen2008efficient,
  title={Efficient simulation budget allocation for selecting an optimal subset},
  author={Chen, Chun-Hung and He, Donghai and Fu, Michael and Lee, Loo Hay},
  journal={INFORMS Journal on Computing},
  volume={20},
  number={4},
  pages={579--595},
  year={2008},
  publisher={INFORMS}
}

@article{ratmann2007using,
  title={Using likelihood-free inference to compare evolutionary dynamics of the protein networks of H. pylori and P. falciparum},
  author={Ratmann, Oliver and J{\o}rgensen, Ole and Hinkley, Trevor and Stumpf, Michael and Richardson, Sylvia and Wiuf, Carsten},
  journal={PLoS Computational Biology},
  volume={3},
  number={11},
  pages={e230},
  year={2007},
  publisher={Public Library of Science}
}

@article{heggland2004estimating,
  title={Estimating functions in indirect inference},
  author={Heggland, Knut and Frigessi, Arnoldo},
  journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)},
  volume={66},
  number={2},
  pages={447--462},
  year={2004},
  publisher={Wiley Online Library}
}

@incollection{blum2010choosing,
  title={Choosing the summary statistics and the acceptance rate in approximate Bayesian computation},
  author={Blum, Michael GB},
  booktitle={Proceedings of COMPSTAT'2010},
  pages={47--56},
  year={2010},
  publisher={Springer}
}

@article{sedki2012contribution,
  title={Contribution to the discussion of Fearnhead and Prangle (2012). Constructing summary statistics for approximate Bayesian computation: Semi-automatic approximate Bayesian computation},
  author={Sedki, MA and Pudlo, P},
  journal={Journal of the Royal Statistical Society: Series B},
  volume={74},
  pages={466--467},
  year={2012}
}

@article{joyce2008approximately,
  title={Approximately sufficient statistics and Bayesian computation},
  author={Joyce, Paul and Marjoram, Paul},
  journal={Statistical applications in genetics and molecular biology},
  volume={7},
  number={1},
  year={2008},
  publisher={De Gruyter}
}

@inproceedings{kalyanakrishnan2010efficient,
  title={Efficient Selection of Multiple Bandit Arms: Theory and Practice.},
  author={Kalyanakrishnan, Shivaram and Stone, Peter},
  booktitle={ICML},
  volume={10},
  pages={511--518},
  year={2010}
}

@book{sisson2018handbook,
  title={Handbook of approximate Bayesian computation},
  author={Sisson, Scott A and Fan, Yanan and Beaumont, Mark},
  year={2018},
  publisher={Chapman and Hall/CRC}
}

@article{sunnaaker2013approximate,
  title={Approximate bayesian computation},
  author={Sunn{\aa}ker, Mikael and Busetto, Alberto Giovanni and Numminen, Elina and Corander, Jukka and Foll, Matthieu and Dessimoz, Christophe},
  journal={PLoS computational biology},
  volume={9},
  number={1},
  pages={e1002803},
  year={2013},
  publisher={Public Library of Science}
}

@article{nunes2015abctools,
  title={abctools: an R package for tuning approximate Bayesian computation analyses},
  author={Nunes, Matthew Alan and Prangle, Dennis},
  journal={The R Journal},
  volume={7},
  number={2},
  pages={189--205},
  year={2015}
}

@misc{simpleabc,
  author = {Morehead, Robert C},
  title = {SimpleABC},
  year = {2016},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/rcmorehead/simpleabc}}
}

@article{Fulcher2013-yw,
	Author = {Fulcher, Ben D and Little, Max A and Jones, Nick S},
	Journal = {J. R. Soc. Interface},
	Keywords = {aim2.bib},
	Number = 83,
	Pages = {20130048},
	Title = {Highly comparative time-series analysis: the empirical structure of time series and their methods},
	Volume = 10,
	Year = 2013}

@inproceedings{vermorel2005multi,
  title={Multi-armed bandit algorithms and empirical evaluation},
  author={Vermorel, Joannes and Mohri, Mehryar},
  booktitle={European conference on machine learning},
  pages={437--448},
  year={2005},
  organization={Springer}
}

@article{ashyraliyev2009systems,
  title={Systems biology: parameter estimation for biochemical models},
  author={Ashyraliyev, Maksat and Fomekong-Nanfack, Yves and Kaandorp, Jaap A and Blom, Joke G},
  journal={The FEBS journal},
  volume={276},
  number={4},
  pages={886--902},
  year={2009},
  publisher={Wiley Online Library}
}

@article{cash1979parameter,
  title={Parameter estimation in astronomy through application of the likelihood ratio},
  author={Cash, Webster},
  journal={The Astrophysical Journal},
  volume={228},
  pages={939--947},
  year={1979}
}

@book{beck1977parameter,
  title={Parameter estimation in engineering and science},
  author={Beck, James Vere and Arnold, Kenneth J},
  year={1977},
  publisher={James Beck}
}

@book{severini2000likelihood,
  title={Likelihood methods in statistics},
  author={Severini, Thomas A},
  year={2000},
  publisher={Oxford University Press}
}

@article{vilar2002mechanisms,
  title={Mechanisms of noise-resistance in genetic oscillators},
  author={Vilar, Jos{\'e} MG and Kueh, Hao Yuan and Barkai, Naama and Leibler, Stanislas},
  journal={Proceedings of the National Academy of Sciences},
  volume={99},
  number={9},
  pages={5988--5992},
  year={2002},
  publisher={National Acad Sciences}
}

@article{abel2016gillespy,
  title={GillesPy: A Python Package for Stochastic Model Building and Simulation},
  author={Abel, John H and Drawert, Brian and Hellander, Andreas and Petzold, Linda R},
  journal={IEEE life sciences letters},
  volume={2},
  number={3},
  pages={35--38},
  year={2016},
  publisher={IEEE}
}

@phdthesis{watkins1989learning,
  title={Learning from delayed rewards},
  author={Watkins, Christopher John Cornish Hellaby},
  year={1989},
  school={King's College, Cambridge}
}

@inproceedings{even2002pac,
  title={PAC bounds for multi-armed bandit and Markov decision processes},
  author={Even-Dar, Eyal and Mannor, Shie and Mansour, Yishay},
  booktitle={International Conference on Computational Learning Theory},
  pages={255--270},
  year={2002},
  organization={Springer}
}

@article{auer2002finite,
  title={Finite-time analysis of the multiarmed bandit problem},
  author={Auer, Peter and Cesa-Bianchi, Nicolo and Fischer, Paul},
  journal={Machine learning},
  volume={47},
  number={2-3},
  pages={235--256},
  year={2002},
  publisher={Springer}
}

@inproceedings{chapelle2011empirical,
  title={An empirical evaluation of thompson sampling},
  author={Chapelle, Olivier and Li, Lihong},
  booktitle={Advances in neural information processing systems},
  pages={2249--2257},
  year={2011}
}

@article{kuleshov2014algorithms,
  title={Algorithms for multi-armed bandit problems},
  author={Kuleshov, Volodymyr and Precup, Doina},
  journal={arXiv preprint arXiv:1402.6028},
  year={2014}
}

@article{zhou2015survey,
  title={A survey on contextual multi-armed bandits},
  author={Zhou, Li},
  journal={arXiv preprint arXiv:1508.03326},
  year={2015}
}

@inproceedings{park2016k2,
  title={K2-ABC: Approximate Bayesian Computation with Kernel Embeddings},
  author={Park, Mijung and Jitkrittum, Wittawat and Sejdinovic, Dino},
  booktitle={Artificial Intelligence and Statistics},
  pages={398--407},
  year={2016}
}

@article{pritchard1999population,
  title={Population growth of human Y chromosomes: a study of Y chromosome microsatellites.},
  author={Pritchard, Jonathan K and Seielstad, Mark T and Perez-Lezaun, Anna and Feldman, Marcus W},
  journal={Molecular biology and evolution},
  volume={16},
  number={12},
  pages={1791--1798},
  year={1999},
  publisher={Oxford University Press}
}

@article{beaumont2009adaptive,
  title={Adaptive approximate Bayesian computation},
  author={Beaumont, Mark A and Cornuet, Jean-Marie and Marin, Jean-Michel and Robert, Christian P},
  journal={Biometrika},
  volume={96},
  number={4},
  pages={983--990},
  year={2009},
  publisher={Oxford University Press}
}

@article{beaumont2002approximate,
  title={Approximate Bayesian computation in population genetics},
  author={Beaumont, Mark A and Zhang, Wenyang and Balding, David J},
  journal={Genetics},
  volume={162},
  number={4},
  pages={2025--2035},
  year={2002},
  publisher={Genetics Soc America}
}

@article{csillery2010approximate,
  title={Approximate Bayesian computation (ABC) in practice},
  author={Csill{\'e}ry, Katalin and Blum, Michael GB and Gaggiotti, Oscar E and Fran{\c{c}}ois, Olivier},
  journal={Trends in ecology \& evolution},
  volume={25},
  number={7},
  pages={410--418},
  year={2010},
  publisher={Elsevier}
}


@article{Lenive2016-zl,
	Abstract = {BACKGROUND: Gene expression is known to be an intrinsically
              stochastic process which can involve single-digit numbers of mRNA
              molecules in a cell at any given time. The modelling of such
              processes calls for the use of exact stochastic simulation
              methods, most notably the Gillespie algorithm. However, this
              stochasticity, also termed ``intrinsic noise'', does not account
              for all the variability between genetically identical cells
              growing in a homogeneous environment. Despite substantial
              experimental efforts, determining appropriate model parameters
              continues to be a challenge. Methods based on approximate
              Bayesian computation can be used to obtain posterior parameter
              distributions given the observed data. However, such inference
              procedures require large numbers of simulations of the model and
              exact stochastic simulation is computationally costly. In this
              work we focus on the specific case of trying to infer model
              parameters describing reaction rates and extrinsic noise on the
              basis of measurements of molecule numbers in individual cells at
              a given time point. RESULTS: To make the problem computationally
              tractable we develop an exact, model-specific, stochastic
              simulation algorithm for the commonly used two-state model of
              gene expression. This algorithm relies on certain assumptions and
              favourable properties of the model to forgo the simulation of the
              whole temporal trajectory of protein numbers in the system,
              instead returning only the number of protein and mRNA molecules
              present in the system at a specified time point. The
              computational gain is proportional to the number of protein
              molecules created in the system and becomes significant for
              systems involving hundreds or thousands of protein molecules.
              CONCLUSIONS: We employ this simulation algorithm with approximate
              Bayesian computation to jointly infer the model's rate and noise
              parameters from published gene expression data. Our analysis
              indicates that for most genes the extrinsic contributions to
              noise will be small to moderate but certainly are non-negligible.},
	Author = {Lenive, Oleg and W Kirk, Paul D and H Stumpf, Michael P},
	Journal = {BMC Syst. Biol.},
	Keywords = {Approximate Bayesian computation; Extrinsic noise; Gene expression; Stochastic simulation},
	Language = {en},
	Month = aug,
	Number = 1,
	Pages = {81},
	Title = {Inferring extrinsic noise from single-cell gene expression data using approximate Bayesian computation},
	Volume = 10,
	Year = 2016}


@article{Lillacci2013-yn,
	Abstract = {MOTIVATION: In the noisy cellular environment, stochastic
              fluctuations at the molecular level manifest as cell-cell
              variability at the population level that is quantifiable using
              high-throughput single-cell measurements. Such variability is
              rich with information about the cell's underlying gene regulatory
              networks, their architecture and the parameters of the
              biochemical reactions at their core. RESULTS: We report a novel
              method, called Inference for Networks of Stochastic Interactions
              among Genes using High-Throughput data (INSIGHT), for
              systematically combining high-throughput time-course flow
              cytometry measurements with computer-generated stochastic
              simulations of candidate gene network models to infer the
              network's stochastic model and all its parameters. By exploiting
              the mathematical relationships between experimental and simulated
              population histograms, INSIGHT achieves scalability, efficiency
              and accuracy while entirely avoiding approximate stochastic
              methods. We demonstrate our method on a synthetic gene network in
              bacteria and show that a detailed mechanistic model of this
              network can be estimated with high accuracy and high efficiency.
              Our method is completely general and can be used to infer models
              of signal-activated gene networks in any organism based solely on
              flow cytometry data and stochastic simulations. AVAILABILITY: A
              free C source code implementing the INSIGHT algorithm, together
              with test data is available from the authors. CONTACT:
              mustafa.khammash@bsse.ethz.ch SUPPLEMENTARY INFORMATION:
              Supplementary data are available at Bioinformatics online.},
	Author = {Lillacci, Gabriele and Khammash, Mustafa},
	Journal = {Bioinformatics},
	Language = {en},
	Month = sep,
	Number = 18,
	Pages = {2311--2319},
	Title = {The signal within the noise: efficient inference of stochastic gene regulation models using fluorescence histograms and stochastic simulations},
	Volume = 29,
	Year = 2013}


@article{Daigle2012-da,
	Abstract = {BACKGROUND: A prerequisite for the mechanistic simulation of a
              biochemical system is detailed knowledge of its kinetic
              parameters. Despite recent experimental advances, the estimation
              of unknown parameter values from observed data is still a
              bottleneck for obtaining accurate simulation results. Many
              methods exist for parameter estimation in deterministic
              biochemical systems; methods for discrete stochastic systems are
              less well developed. Given the probabilistic nature of stochastic
              biochemical models, a natural approach is to choose parameter
              values that maximize the probability of the observed data with
              respect to the unknown parameters, a.k.a. the maximum likelihood
              parameter estimates (MLEs). MLE computation for all but the
              simplest models requires the simulation of many system
              trajectories that are consistent with experimental data. For
              models with unknown parameters, this presents a computational
              challenge, as the generation of consistent trajectories can be an
              extremely rare occurrence. RESULTS: We have developed Monte Carlo
              Expectation-Maximization with Modified Cross-Entropy Method
              (MCEM(2)): an accelerated method for calculating MLEs that
              combines advances in rare event simulation with a computationally
              efficient version of the Monte Carlo expectation-maximization
              (MCEM) algorithm. Our method requires no prior knowledge
              regarding parameter values, and it automatically provides a
              multivariate parameter uncertainty estimate. We applied the
              method to five stochastic systems of increasing complexity,
              progressing from an analytically tractable pure-birth model to a
              computationally demanding model of yeast-polarization. Our
              results demonstrate that MCEM(2) substantially accelerates MLE
              computation on all tested models when compared to a stand-alone
              version of MCEM. Additionally, we show how our method identifies
              parameter values for certain classes of models more accurately
              than two recently proposed computationally efficient methods.
              CONCLUSIONS: This work provides a novel, accelerated version of a
              likelihood-based parameter estimation method that can be readily
              applied to stochastic biochemical systems. In addition, our
              results suggest opportunities for added efficiency improvements
              that will further enhance our ability to mechanistically simulate
              biological processes.},
	Author = {Daigle, Jr, Bernie J and Roh, Min K and Petzold, Linda R and Niemi, Jarad},
	Journal = {BMC Bioinformatics},
	Language = {en},
	Month = may,
	Pages = {68},
	Title = {Accelerated maximum likelihood parameter estimation for stochastic biochemical systems},
	Volume = 13,
	Year = 2012}


@article{Gillespie1976-eq,
	Author = {Gillespie, Daniel T},
	Journal = {J. Comput. Phys.},
	Keywords = {vr.bib},
	Pages = {403--434},
	Title = {A general method for numerically simulating the stochastic time evolution of coupled chemical reacting systems},
	Volume = 22,
	Year = 1976}

@article{Elowitz2002-ho,
	Author = {Elowitz, Michael B and Levine, Arnold J and Siggia, Eric D and Swain, Peter S},
	Journal = {Science},
	Language = {en},
	Month = aug,
	Number = 5584,
	Pages = {1183--1186},
	Title = {Stochastic gene expression in a single cell},
	Volume = 297,
	Year = 2002}

@article{prangle2015summary,
  title={Summary statistics in approximate Bayesian computation},
  author={Prangle, Dennis},
  journal={arXiv preprint arXiv:1512.05633},
  year={2015}
}

@incollection{robbins1985some,
  title={Some aspects of the sequential design of experiments},
  author={Robbins, Herbert},
  booktitle={Herbert Robbins Selected Papers},
  pages={169--177},
  year={1985},
  publisher={Springer}
}

@article{christ2016distributed,
  title={Distributed and parallel time series feature extraction for industrial big data applications},
  author={Christ, Maximilian and Kempa-Liehr, Andreas W and Feindt, Michael},
  journal={arXiv preprint arXiv:1610.07717},
  year={2016}
}

@article{fearnhead2012constructing,
  title={Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation},
  author={Fearnhead, Paul and Prangle, Dennis},
  journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)},
  volume={74},
  number={3},
  pages={419--474},
  year={2012},
  publisher={Wiley Online Library}
}

@article{nunes2010optimal,
  title={On optimal selection of summary statistics for approximate Bayesian computation},
  author={Nunes, Matthew A and Balding, David J},
  journal={Statistical applications in genetics and molecular biology},
  volume={9},
  number={1},
  year={2010},
  publisher={De Gruyter}
}

@article{barnes2012considerate,
  title={Considerate approaches to constructing summary statistics for ABC model selection},
  author={Barnes, Chris P and Filippi, Sarah and Stumpf, Michael PH and Thorne, Thomas},
  journal={Statistics and Computing},
  volume={22},
  number={6},
  pages={1181--1197},
  year={2012},
  publisher={Springer}
}

@article{prangle2014semi,
  title={Semi-automatic selection of summary statistics for ABC model choice},
  author={Prangle, Dennis and Fearnhead, Paul and Cox, Murray P and Biggs, Patrick J and French, Nigel P},
  journal={Statistical applications in genetics and molecular biology},
  volume={13},
  number={1},
  pages={67--82},
  year={2014},
  publisher={De Gruyter}
}

@incollection{blum2010choosing,
  title={Choosing the summary statistics and the acceptance rate in approximate Bayesian computation},
  author={Blum, Michael GB},
  booktitle={Proceedings of COMPSTAT'2010},
  pages={47--56},
  year={2010},
  publisher={Springer}
}

@article{sedki2012contribution,
  title={Contribution to the discussion of Fearnhead and Prangle (2012). Constructing summary statistics for approximate Bayesian computation: Semi-automatic approximate Bayesian computation},
  author={Sedki, MA and Pudlo, P},
  journal={Journal of the Royal Statistical Society: Series B},
  volume={74},
  pages={466--467},
  year={2012}
}

@article{aeschbacher2012novel,
  title={A novel approach for choosing summary statistics in approximate Bayesian computation},
  author={Aeschbacher, Simon and Beaumont, Mark A and Futschik, Andreas},
  journal={Genetics},
  volume={192},
  number={3},
  pages={1027--1047},
  year={2012},
  publisher={Genetics Soc America}
}

@article{fu1997estimating,
  title={Estimating the age of the common ancestor of a sample of DNA sequences.},
  author={Fu, Yun-Xin and Li, Wen-Hsiung},
  journal={Molecular biology and evolution},
  volume={14},
  number={2},
  pages={195--199},
  year={1997}
}

@article{wegmann2009efficient,
  title={Efficient approximate Bayesian computation coupled with Markov chain Monte Carlo without likelihood},
  author={Wegmann, Daniel and Leuenberger, Christoph and Excoffier, Laurent},
  journal={Genetics},
  volume={182},
  number={4},
  pages={1207--1218},
  year={2009},
  publisher={Genetics Soc America}
}

@article{blum2013comparative,
  title={A comparative review of dimension reduction methods in approximate Bayesian computation},
  author={Blum, Michael GB and Nunes, Maria Antonieta and Prangle, Dennis and Sisson, Scott A and others},
  journal={Statistical Science},
  volume={28},
  number={2},
  pages={189--208},
  year={2013},
  publisher={Institute of Mathematical Statistics}
}

@article{joyce2008approximately,
  title={Approximately sufficient statistics and Bayesian computation},
  author={Joyce, Paul and Marjoram, Paul},
  journal={Statistical applications in genetics and molecular biology},
  volume={7},
  number={1},
  year={2008},
  publisher={De Gruyter}
}

@article{barber2015rate,
  title={The rate of convergence for approximate Bayesian computation},
  author={Barber, Stuart and Voss, Jochen and Webster, Mark and others},
  journal={Electronic Journal of Statistics},
  volume={9},
  number={1},
  pages={80--105},
  year={2015},
  publisher={The Institute of Mathematical Statistics and the Bernoulli Society}
}

@article{lintusaari2017fundamentals,
  title={Fundamentals and recent developments in approximate Bayesian computation},
  author={Lintusaari, Jarno and Gutmann, Michael U and Dutta, Ritabrata and Kaski, Samuel and Corander, Jukka},
  journal={Systematic biology},
  volume={66},
  number={1},
  pages={e66--e82},
  year={2017},
  publisher={Oxford University Press}
}

@article{cornuet2008inferring,
  title={Inferring population history with DIY ABC: a user-friendly approach to approximate Bayesian computation},
  author={Cornuet, Jean-Marie and Santos, Filipe and Beaumont, Mark A and Robert, Christian P and Marin, Jean-Michel and Balding, David J and Guillemaud, Thomas and Estoup, Arnaud},
  journal={Bioinformatics},
  volume={24},
  number={23},
  pages={2713--2719},
  year={2008},
  publisher={Oxford University Press}
}


@article{diggle1984monte,
  title={Monte Carlo methods of inference for implicit statistical models},
  author={Diggle, Peter J and Gratton, Richard J},
  journal={Journal of the Royal Statistical Society. Series B (Methodological)},
  pages={193--227},
  year={1984},
  publisher={JSTOR}
}

@article{drawert2016stochastic,
  title={Stochastic simulation service: bridging the gap between the computational expert and the biologist},
  author={Drawert, Brian and Hellander, Andreas and Bales, Ben and Banerjee, Debjani and Bellesia, Giovanni and Daigle Jr, Bernie J and Douglas, Geoffrey and Gu, Mengyuan and Gupta, Anand and Hellander, Stefan and others},
  journal={PLoS computational biology},
  volume={12},
  number={12},
  pages={e1005220},
  year={2016},
  publisher={Public Library of Science}
}



@Article{Wegmann2009,
author={Wegmann, Daniel
and Leuenberger, Christoph
and Excoffier, Laurent},
title={Efficient approximate Bayesian computation coupled with Markov chain Monte Carlo without likelihood},
journal={Genetics},
year={2009},
month={Aug},
publisher={Genetics Society of America},
volume={182},
number={4},
pages={1207-1218},
keywords={Africa},
keywords={Bayes Theorem},
keywords={Emigration and Immigration},
keywords={Humans},
keywords={Likelihood Functions},
keywords={Markov Chains},
keywords={*Models, Genetic},
keywords={Monte Carlo Method},
keywords={*Population Dynamics},
keywords={*Probability},
keywords={Statistics as Topic},
abstract={Approximate Bayesian computation (ABC) techniques permit inferences in complex demographic models, but are computationally inefficient. A Markov chain Monte Carlo (MCMC) approach has been proposed (Marjoram et al. 2003), but it suffers from computational problems and poor mixing. We propose several methodological developments to overcome the shortcomings of this MCMC approach and hence realize substantial computational advances over standard ABC. The principal idea is to relax the tolerance within MCMC to permit good mixing, but retain a good approximation to the posterior by a combination of subsampling the output and regression adjustment. We also propose to use a partial least-squares (PLS) transformation to choose informative statistics. The accuracy of our approach is examined in the case of the divergence of two populations with and without migration. In that case, our ABC-MCMC approach needs considerably lower computation time to reach the same accuracy than conventional ABC. We then apply our method to a more complex case with the estimation of divergence times and migration rates between three African populations.},
note={19506307[pmid]},
note={PMC2728860[pmcid]},
note={genetics.109.102509[PII]},
issn={1943-2631},
doi={10.1534/genetics.109.102509},
url={https://www.ncbi.nlm.nih.gov/pubmed/19506307}
}


@Inbook{Paul2008,
author={Paul, Joyce
and Paul, Marjoram},
chapter={Approximately Sufficient Statistics and Bayesian Computation},
title={sagmb},
year={2008},
month={2019},
day={05-20T11:06:11.077+02:00},
volume={7},
note={1},
issn={15446115},
doi={10.2202/1544-6115.1389},
url={https://www.degruyter.com/view/j/sagmb.2008.7.1/sagmb.2008.7.1.1389/sagmb.2008.7.1.1389.xml}
}

@ARTICLE{2010arXiv1004.1112F,
       author = {{Fearnhead}, Paul and {Prangle}, Dennis},
        title = "{Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC}",
      journal = {arXiv e-prints},
     keywords = {Statistics - Methodology, Statistics - Computation},
         year = "2010",
        month = "Apr",
          eid = {arXiv:1004.1112},
        pages = {arXiv:1004.1112},
archivePrefix = {arXiv},
       eprint = {1004.1112},
 primaryClass = {stat.ME},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2010arXiv1004.1112F},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

@article {Vilar5988,
	author = {Vilar, Jos{\'e} M. G. and Kueh, Hao Yuan and Barkai, Naama and Leibler, Stanislas},
	title = {Mechanisms of noise-resistance in genetic oscillators},
	volume = {99},
	number = {9},
	pages = {5988--5992},
	year = {2002},
	doi = {10.1073/pnas.092133899},
	publisher = {National Academy of Sciences},
	abstract = {A wide range of organisms use circadian clocks to keep internal sense of daily time and regulate their behavior accordingly. Most of these clocks use intracellular genetic networks based on positive and negative regulatory elements. The integration of these {\textquotedblleft}circuits{\textquotedblright} at the cellular level imposes strong constraints on their functioning and design. Here, we study a recently proposed model [Barkai, N. \&amp; Leibler, S. (2000) Nature (London), 403, 267{\textendash}268] that incorporates just the essential elements found experimentally. We show that this type of oscillator is driven mainly by two elements: the concentration of a repressor protein and the dynamics of an activator protein forming an inactive complex with the repressor. Thus, the clock does not need to rely on mRNA dynamics to oscillate, which makes it especially resistant to fluctuations. Oscillations can be present even when the time average of the number of mRNA molecules goes below one. Under some conditions, this oscillator is not only resistant to but, paradoxically, also enhanced by the intrinsic biochemical noise.},
	issn = {0027-8424},
	URL = {https://www.pnas.org/content/99/9/5988},
	eprint = {https://www.pnas.org/content/99/9/5988.full.pdf},
	journal = {Proceedings of the National Academy of Sciences}
}

@Article{Gillespie1977,
author={Gillespie, Daniel T.},
title={Exact stochastic simulation of coupled chemical reactions},
journal={The Journal of Physical Chemistry},
year={1977},
month={Dec},
day={01},
publisher={American Chemical Society},
volume={81},
number={25},
pages={2340-2361},
issn={0022-3654},
doi={10.1021/j100540a008},
url={https://doi.org/10.1021/j100540a008}
}

@inproceedings{2019arXiv190110230W,
  title={Partially Exchangeable Networks and Architectures for Learning Summary Statistics in Approximate Bayesian Computation},
  author={Wiqvist, Samuel and Mattei, Pierre-Alexandre and Picchini, Umberto and Frellsen, Jes},
  booktitle={International Conference on Machine Learning},
  pages={6798--6807},
  year={2019}
}

@article{goh2008burstiness,
  title={Burstiness and memory in complex systems},
  author={Goh, K-I and Barab{\'a}si, A-L},
  journal={EPL (Europhysics Letters)},
  volume={81},
  number={4},
  pages={48002},
  year={2008},
  publisher={IOP Publishing}
}

@incollection{lecun1989generalization,
  title={Generalization and network design strategies},
  author={LeCun, Yann and others},
  booktitle={Connectionism in perspective},
  volume={19},
  year={1989},
  publisher={Citeseer}
}

@article{joyce2008approximately,
  title={Approximately sufficient statistics and Bayesian computation},
  author={Joyce, Paul and Marjoram, Paul},
  journal={Statistical applications in genetics and molecular biology},
  volume={7},
  number={1},
  year={2008},
  publisher={De Gruyter}
}

@inproceedings{zhou1988computation,
  title={Computation of optical flow using a neural network},
  author={Zhou, Yi-Tong and Chellappa, Rama},
  booktitle={IEEE International Conference on Neural Networks},
  volume={1998},
  pages={71--78},
  year={1988}
}

@inproceedings{ryder2018black,
  title={Black-Box Variational Inference for Stochastic Differential Equations},
  author={Ryder, Tom and Golightly, Andrew and McGough, A Stephen and Prangle, Dennis},
  booktitle={International Conference on Machine Learning},
  pages={4423--4432},
  year={2018}
}

@article{golightly2011bayesian,
  title={Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo},
  author={Golightly, Andrew and Wilkinson, Darren J},
  journal={Interface focus},
  volume={1},
  number={6},
  pages={807--820},
  year={2011},
  publisher={The Royal Society}
}

@article{HDR,
author = { Rob J.   Hyndman },
title = {Computing and Graphing Highest Density Regions},
journal = {The American Statistician},
volume = {50},
number = {2},
pages = {120-126},
year  = {1996},
publisher = {Taylor & Francis},
doi = {10.1080/00031305.1996.10474359},

URL = {
        https://www.tandfonline.com/doi/abs/10.1080/00031305.1996.10474359
},
eprint = {
        https://www.tandfonline.com/doi/pdf/10.1080/00031305.1996.10474359
    }
}


@book{goodfellow2016deep,
  title={Deep learning},
  author={Goodfellow, Ian and Bengio, Yoshua and Courville, Aaron},
  year={2016},
  publisher={MIT press}
}

@inproceedings{krizhevsky2012imagenet,
  title={Imagenet classification with deep convolutional neural networks},
  author={Krizhevsky, Alex and Sutskever, Ilya and Hinton, Geoffrey E},
  booktitle={Advances in neural information processing systems},
  pages={1097--1105},
  year={2012}
}

@article{DBLP:journals/corr/ZaheerKRPSS17,
  author    = {Manzil Zaheer and
               Satwik Kottur and
               Siamak Ravanbakhsh and
               Barnab{\'{a}}s P{\'{o}}czos and
               Ruslan Salakhutdinov and
               Alexander J. Smola},
  title     = {Deep Sets},
  journal   = {CoRR},
  volume    = {abs/1703.06114},
  year      = {2017},
  url       = {http://arxiv.org/abs/1703.06114},
  archivePrefix = {arXiv},
  eprint    = {1703.06114},
  timestamp = {Mon, 13 Aug 2018 16:46:16 +0200},
  biburl    = {https://dblp.org/rec/bib/journals/corr/ZaheerKRPSS17},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}
@article{lotka,
author = {Lotka, Alfred J.},
title = {UNDAMPED OSCILLATIONS DERIVED FROM THE LAW OF MASS ACTION.},
journal = {Journal of the American Chemical Society},
volume = {42},
number = {8},
pages = {1595-1599},
year = {1920},
doi = {10.1021/ja01453a010},

URL = {
        https://doi.org/10.1021/ja01453a010

},
eprint = {
        https://doi.org/10.1021/ja01453a010

}

}
@article{gillespy_2017,
	title = {{GillesPy}: {A} {Python} {Package} for {Stochastic} {Model} {Building} and {Simulation}},
	issn = {2332-7685},
	shorttitle = {{GillesPy}},
	url = {http://ieeexplore.ieee.org/document/7843624/},
	doi = {10.1109/LLS.2017.2652448},
	urldate = {2017-02-22},
	journal = {IEEE Life Sciences Letters},
	author = {Abel, John and Drawert, Brian and Hellander, Andreas and Petzold, Linda R},
	year = {2017},
	pages = {1--1}
}
@misc{tensorflow2015-whitepaper,
title={ {TensorFlow}: Large-Scale Machine Learning on Heterogeneous Systems},
url={https://www.tensorflow.org/},
note={Software available from tensorflow.org},
author={
    Mart\'{\i}n~Abadi and
    Ashish~Agarwal and
    Paul~Barham and
    Eugene~Brevdo and
    Zhifeng~Chen and
    Craig~Citro and
    Greg~S.~Corrado and
    Andy~Davis and
    Jeffrey~Dean and
    Matthieu~Devin and
    Sanjay~Ghemawat and
    Ian~Goodfellow and
    Andrew~Harp and
    Geoffrey~Irving and
    Michael~Isard and
    Yangqing Jia and
    Rafal~Jozefowicz and
    Lukasz~Kaiser and
    Manjunath~Kudlur and
    Josh~Levenberg and
    Dandelion~Man\'{e} and
    Rajat~Monga and
    Sherry~Moore and
    Derek~Murray and
    Chris~Olah and
    Mike~Schuster and
    Jonathon~Shlens and
    Benoit~Steiner and
    Ilya~Sutskever and
    Kunal~Talwar and
    Paul~Tucker and
    Vincent~Vanhoucke and
    Vijay~Vasudevan and
    Fernanda~Vi\'{e}gas and
    Oriol~Vinyals and
    Pete~Warden and
    Martin~Wattenberg and
    Martin~Wicke and
    Yuan~Yu and
    Xiaoqiang~Zheng},
  year={2015},
}

@article{christ2018time,
  title={Time series feature extraction on basis of scalable hypothesis tests (tsfresh--a python package)},
  author={Christ, Maximilian and Braun, Nils and Neuffer, Julius and Kempa-Liehr, Andreas W},
  journal={Neurocomputing},
  volume={307},
  pages={72--77},
  year={2018},
  publisher={Elsevier}
}
toArxiv.bbl
\begin{thebibliography}{39}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{doi: #1}\else
  \providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi

\bibitem[Abel et~al.(2017)Abel, Drawert, Hellander, and Petzold]{gillespy_2017}
Abel, J., Drawert, B., Hellander, A., and Petzold, L.~R.
\newblock {GillesPy}: {A} {Python} {Package} for {Stochastic} {Model}
  {Building} and {Simulation}.
\newblock \emph{IEEE Life Sciences Letters}, pp.\  1--1, 2017.
\newblock ISSN 2332-7685.
\newblock \doi{10.1109/LLS.2017.2652448}.
\newblock URL \url{http://ieeexplore.ieee.org/document/7843624/}.

\bibitem[Ahmad et~al.(2012)Ahmad, Halawani, and Albidewi]{AHMAD_RvC}
Ahmad, A., Halawani, S.~M., and Albidewi, I.~A.
\newblock Novel ensemble methods for regression via classification problems.
\newblock \emph{Expert Systems with Applications}, 39\penalty0 (7):\penalty0
  6396--6401, 2012.
\newblock ISSN 0957-4174.
\newblock \doi{https://doi.org/10.1016/j.eswa.2011.12.029}.
\newblock URL
  \url{https://www.sciencedirect.com/science/article/pii/S0957417411017003}.

\bibitem[{\AA}kesson et~al.(2020){\AA}kesson, Singh, Wrede, and
  Hellander]{aakesson2020convolutional}
{\AA}kesson, M., Singh, P., Wrede, F., and Hellander, A.
\newblock Convolutional neural networks as summary statistics for approximate
  bayesian computation.
\newblock \emph{arXiv preprint arXiv:2001.11760}, 2020.

\bibitem[Blundell et~al.(2015)Blundell, Cornebise, Kavukcuoglu, and
  Wierstra]{Bayesbybackprop}
Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D.
\newblock Weight uncertainty in neural network.
\newblock In \emph{Proceedings of the 32nd International Conference on Machine
  Learning}, volume~37 of \emph{Proceedings of Machine Learning Research}, pp.\
   1613--1622, Lille, France, 07--09 Jul 2015. PMLR.
\newblock URL \url{http://proceedings.mlr.press/v37/blundell15.html}.

\bibitem[Chen et~al.(2020)Chen, Zhang, Gutmann, Courville, and
  Zhu]{chen2020neural}
Chen, Y., Zhang, D., Gutmann, M., Courville, A., and Zhu, Z.
\newblock Neural approximate sufficient statistics for implicit models.
\newblock \emph{arXiv preprint arXiv:2010.10079}, 2020.

\bibitem[Christ et~al.(2018)Christ, Braun, Neuffer, and
  Kempa-Liehr]{christ2018time}
Christ, M., Braun, N., Neuffer, J., and Kempa-Liehr, A.~W.
\newblock Time series feature extraction on basis of scalable hypothesis tests
  (tsfresh--a python package).
\newblock \emph{Neurocomputing}, 307:\penalty0 72--77, 2018.

\bibitem[Cranmer et~al.(2020)Cranmer, Brehmer, and Louppe]{Cranmer_frontierSBI}
Cranmer, K., Brehmer, J., and Louppe, G.
\newblock The frontier of simulation-based inference.
\newblock \emph{Proceedings of the National Academy of Sciences}, 117\penalty0
  (48):\penalty0 30055--30062, 2020.
\newblock ISSN 0027-8424.
\newblock \doi{10.1073/pnas.1912789117}.
\newblock URL \url{https://www.pnas.org/content/117/48/30055}.

\bibitem[Del~Moral et~al.(2012)Del~Moral, Doucet, and Jasra]{del2012adaptive}
Del~Moral, P., Doucet, A., and Jasra, A.
\newblock An adaptive sequential monte carlo method for approximate bayesian
  computation.
\newblock \emph{Statistics and computing}, 22\penalty0 (5):\penalty0
  1009--1020, 2012.

\bibitem[Dinev \& Gutmann(2018)Dinev and Gutmann]{dinev2018dynamic}
Dinev, T. and Gutmann, M.~U.
\newblock Dynamic likelihood-free inference via ratio estimation (dire).
\newblock \emph{arXiv preprint arXiv:1810.09899}, 2018.

\bibitem[Durkan et~al.(2019)Durkan, Bekasov, Murray, and Papamakarios]{NSF}
Durkan, C., Bekasov, A., Murray, I., and Papamakarios, G.
\newblock Neural spline flows.
\newblock In \emph{Advances in Neural Information Processing Systems},
  volume~32, pp.\  7511--7522. Curran Associates, Inc., 2019.
\newblock URL
  \url{https://proceedings.neurips.cc/paper/2019/file/7ac71d433f282034e088473244df8c02-Paper.pdf}.

\bibitem[Durkan et~al.(2020)Durkan, Murray, and
  Papamakarios]{SNRE_b_constrative}
Durkan, C., Murray, I., and Papamakarios, G.
\newblock On contrastive learning for likelihood-free inference.
\newblock In \emph{Proceedings of the 37th International Conference on Machine
  Learning}, volume 119 of \emph{Proceedings of Machine Learning Research},
  pp.\  2771--2781. PMLR, 13--18 Jul 2020.
\newblock URL \url{http://proceedings.mlr.press/v119/durkan20a.html}.

\bibitem[Engblom et~al.(2020)Engblom, Eriksson, and
  Widgren]{engblom2020bayesian}
Engblom, S., Eriksson, R., and Widgren, S.
\newblock Bayesian epidemiological modeling over high-resolution network data.
\newblock \emph{Epidemics}, 32:\penalty0 100399, 2020.

\bibitem[Fearnhead \& Prangle(2012)Fearnhead and
  Prangle]{fearnhead2012constructing}
Fearnhead, P. and Prangle, D.
\newblock Constructing summary statistics for approximate bayesian computation:
  semi-automatic approximate bayesian computation.
\newblock \emph{Journal of the Royal Statistical Society: Series B (Statistical
  Methodology)}, 74\penalty0 (3):\penalty0 419--474, 2012.

\bibitem[Gillespie(1977)]{gillespie1977exact}
Gillespie, D.~T.
\newblock Exact stochastic simulation of coupled chemical reactions.
\newblock \emph{The journal of physical chemistry}, 81\penalty0 (25):\penalty0
  2340--2361, 1977.

\bibitem[Greenberg et~al.(2019{\natexlab{a}})Greenberg, Nonnenmacher, and
  Macke]{APT_2019}
Greenberg, D., Nonnenmacher, M., and Macke, J.
\newblock Automatic posterior transformation for likelihood-free inference.
\newblock In \emph{Proceedings of the 36th International Conference on Machine
  Learning}, volume~97 of \emph{Proceedings of Machine Learning Research}, pp.\
   2404--2414. PMLR, 09--15 Jun 2019{\natexlab{a}}.
\newblock URL \url{http://proceedings.mlr.press/v97/greenberg19a.html}.

\bibitem[Greenberg et~al.(2019{\natexlab{b}})Greenberg, Nonnenmacher, and
  Macke]{greenberg2019automatic}
Greenberg, D., Nonnenmacher, M., and Macke, J.
\newblock Automatic posterior transformation for likelihood-free inference.
\newblock In \emph{International Conference on Machine Learning}, pp.\
  2404--2414. PMLR, 2019{\natexlab{b}}.

\bibitem[Gretton et~al.(2012)Gretton, Borgwardt, Rasch, Sch{\"o}lkopf, and
  Smola]{Scholkopf_MMD}
Gretton, A., Borgwardt, K., Rasch, M., Sch{\"o}lkopf, B., and Smola, A.
\newblock A kernel two-sample test.
\newblock \emph{Journal of Machine Learning Research}, 13:\penalty0 723--773,
  March 2012.

\bibitem[Hermans et~al.(2020)Hermans, Begy, and Louppe]{SNRE_a}
Hermans, J., Begy, V., and Louppe, G.
\newblock Likelihood-free {MCMC} with amortized approximate ratio estimators.
\newblock In \emph{Proceedings of the 37th International Conference on Machine
  Learning}, volume 119 of \emph{Proceedings of Machine Learning Research},
  pp.\  4239--4248. PMLR, 13--18 Jul 2020.
\newblock URL \url{http://proceedings.mlr.press/v119/hermans20a.html}.

\bibitem[Hyndman(1996)]{HDR}
Hyndman, R.~J.
\newblock Computing and graphing highest density regions.
\newblock \emph{The American Statistician}, 50\penalty0 (2):\penalty0 120--126,
  1996.
\newblock \doi{10.1080/00031305.1996.10474359}.
\newblock URL
  \url{https://www.tandfonline.com/doi/abs/10.1080/00031305.1996.10474359}.

\bibitem[Jiang et~al.(2017)Jiang, Wu, Zheng, and Wong]{jiang2017learning}
Jiang, B., Wu, T.-y., Zheng, C., and Wong, W.~H.
\newblock Learning summary statistic for approximate bayesian computation via
  deep neural network.
\newblock \emph{Statistica Sinica}, pp.\  1595--1618, 2017.

\bibitem[Levina \& Bickel(2001)Levina and Bickel]{levina2001earth}
Levina, E. and Bickel, P.
\newblock The earth mover's distance is the mallows distance: Some insights
  from statistics.
\newblock In \emph{Proceedings Eighth IEEE International Conference on Computer
  Vision. ICCV 2001}, volume~2, pp.\  251--256. IEEE, 2001.

\bibitem[Lotka(1920)]{lotka}
Lotka, A.~J.
\newblock Undamped oscillations derived from the law of mass action.
\newblock \emph{Journal of the American Chemical Society}, 42\penalty0
  (8):\penalty0 1595--1599, 1920.
\newblock \doi{10.1021/ja01453a010}.
\newblock URL \url{https://doi.org/10.1021/ja01453a010}.

\bibitem[Lueckmann et~al.(2017)Lueckmann, Gon{\c{c}}alves, Bassetto, {\"O}cal,
  Nonnenmacher, and Macke]{SNPE_B_lueckmann2017flexible}
Lueckmann, J.-M., Gon{\c{c}}alves, P.~J., Bassetto, G., {\"O}cal, K.,
  Nonnenmacher, M., and Macke, J.~H.
\newblock Flexible statistical inference for mechanistic models of neural
  dynamics.
\newblock In \emph{Proceedings of the 31st International Conference on Neural
  Information Processing Systems}, pp.\  1289--1299, 2017.

\bibitem[Lueckmann et~al.(2019)Lueckmann, Bassetto, Karaletsos, and
  Macke]{lueckmann2019likelihood}
Lueckmann, J.-M., Bassetto, G., Karaletsos, T., and Macke, J.~H.
\newblock Likelihood-free inference with emulator networks.
\newblock In \emph{Symposium on Advances in Approximate Bayesian Inference},
  pp.\  32--53. PMLR, 2019.

\bibitem[Lueckmann et~al.(2021)Lueckmann, Boelts, Greenberg, Gon{\c{c}}alves,
  and Macke]{lueckmann2021benchmarking}
Lueckmann, J.-M., Boelts, J., Greenberg, D.~S., Gon{\c{c}}alves, P.~J., and
  Macke, J.~H.
\newblock Benchmarking simulation-based inference.
\newblock \emph{arXiv preprint arXiv:2101.04653}, 2021.

\bibitem[Marin et~al.(2012)Marin, Pudlo, Robert, and
  Ryder]{marin2012approximate}
Marin, J.-M., Pudlo, P., Robert, C.~P., and Ryder, R.~J.
\newblock Approximate bayesian computational methods.
\newblock \emph{Statistics and Computing}, 22\penalty0 (6):\penalty0
  1167--1180, 2012.

\bibitem[Owen et~al.(2015)Owen, Wilkinson, and Gillespie]{owen2015scalable}
Owen, J., Wilkinson, D.~J., and Gillespie, C.~S.
\newblock Scalable inference for markov processes with intractable likelihoods.
\newblock \emph{Statistics and Computing}, 25\penalty0 (1):\penalty0 145--156,
  2015.

\bibitem[Papamakarios \& Murray(2016)Papamakarios and
  Murray]{SNPE_A_Papamakarios}
Papamakarios, G. and Murray, I.
\newblock Fast {$\epsilon$}-free inference of simulation models with bayesian
  conditional density estimation.
\newblock In \emph{Advances in Neural Information Processing Systems},
  volume~29, pp.\  1028--1036. Curran Associates, Inc., 2016.
\newblock URL
  \url{https://proceedings.neurips.cc/paper/2016/file/6aca97005c68f1206823815f66102863-Paper.pdf}.

\bibitem[Papamakarios et~al.(2017)Papamakarios, Pavlakou, and
  Murray]{papamakarios2017masked}
Papamakarios, G., Pavlakou, T., and Murray, I.
\newblock Masked autoregressive flow for density estimation.
\newblock In \emph{Proceedings of the 31st International Conference on Neural
  Information Processing Systems}, pp.\  2335--2344, 2017.

\bibitem[Papamakarios et~al.(2019)Papamakarios, Sterratt, and
  Murray]{papamakarios2019sequential}
Papamakarios, G., Sterratt, D., and Murray, I.
\newblock Sequential neural likelihood: Fast likelihood-free inference with
  autoregressive flows.
\newblock In \emph{The 22nd International Conference on Artificial Intelligence
  and Statistics}, pp.\  837--848. PMLR, 2019.

\bibitem[Prangle et~al.(2017)]{prangle2017adapting}
Prangle, D. et~al.
\newblock Adapting the abc distance function.
\newblock \emph{Bayesian Analysis}, 12\penalty0 (1):\penalty0 289--309, 2017.

\bibitem[Sadouk(2019)]{sadouk2019cnn}
Sadouk, L.
\newblock Cnn approaches for time series classification.
\newblock In \emph{Time Series Analysis-Data, Methods, and Applications}, pp.\
  1--23. IntechOpen, 2019.

\bibitem[Sisson et~al.(2007)Sisson, Fan, and Tanaka]{sisson2007sequential}
Sisson, S.~A., Fan, Y., and Tanaka, M.~M.
\newblock Sequential monte carlo without likelihoods.
\newblock \emph{Proceedings of the National Academy of Sciences}, 104\penalty0
  (6):\penalty0 1760--1765, 2007.

\bibitem[Sisson et~al.(2018)Sisson, Fan, and Beaumont]{sisson2018handbook}
Sisson, S.~A., Fan, Y., and Beaumont, M.
\newblock \emph{Handbook of approximate Bayesian computation}.
\newblock Chapman and Hall/CRC, 2018.

\bibitem[Tejero-Cantero et~al.(2020)Tejero-Cantero, Boelts, Deistler,
  Lueckmann, Durkan, Gon{\c{c}}alves, Greenberg, and Macke]{tejero2020sbi}
Tejero-Cantero, A., Boelts, J., Deistler, M., Lueckmann, J.-M., Durkan, C.,
  Gon{\c{c}}alves, P.~J., Greenberg, D.~S., and Macke, J.~H.
\newblock Sbi: a toolkit for simulation-based inference.
\newblock \emph{Journal of Open Source Software}, 5\penalty0 (52):\penalty0
  2505, 2020.

\bibitem[Torgo \& Gama(1996)Torgo and Gama]{torgo1996regression}
Torgo, L. and Gama, J.
\newblock Regression by classification.
\newblock In \emph{Brazilian symposium on artificial intelligence}, pp.\
  51--60. Springer, 1996.

\bibitem[Weiss \& Indurkhya(1995)Weiss and Indurkhya]{weiss1995rule}
Weiss, S.~M. and Indurkhya, N.
\newblock Rule-based machine learning methods for functional prediction.
\newblock \emph{Journal of Artificial Intelligence Research}, 3:\penalty0
  383--403, 1995.

\bibitem[Wiqvist et~al.(2019)Wiqvist, Mattei, Picchini, and
  Frellsen]{wiqvist2019partially}
Wiqvist, S., Mattei, P.-A., Picchini, U., and Frellsen, J.
\newblock Partially exchangeable networks and architectures for learning
  summary statistics in approximate bayesian computation.
\newblock In \emph{International Conference on Machine Learning}, pp.\
  6798--6807. PMLR, 2019.

\bibitem[Wood(2010)]{wood2010statistical}
Wood, S.~N.
\newblock Statistical inference for noisy nonlinear ecological dynamic systems.
\newblock \emph{Nature}, 466\penalty0 (7310):\penalty0 1102--1104, 2010.

\end{thebibliography}
