Here we turn to discuss two approximate methods that can, under certain
circumstances, be used instead of a full-fledged fit to estimate the
impact on the PDFs of new experimental measurements.
%
These two methods go under the name of
the Bayesian reweighting of Monte Carlo
replicas~\cite{Ball:2011gg,Ball:2010gb} and the profiling
of Hessian sets~\cite{Camarda:2015zba}.
%
The main advantage of these two techniques is that they can be used  to quantify
the impact of new experiments on a pre-existing fit based only on publicly available
information, in particular the {\tt LHAPDF} PDF grids, and thus
such studies can be carried out outside the PDF fitting collaborations.

On the other hand, these methods have a number of limitations, for instance,
they can fail if the impact of the new experiments is too large, and moreover
are not able to account for the effect of methodological changes, such as the input
PDF parametrization, or of modifications in the theoretical calculations.
%
Therefore, they represent an important addition to the PDF fitting toolbox,
but some care should be taken when using them and when interpreting
their results.

\subsubsection{Bayesian Monte Carlo reweighting}

The Bayesian reweighting
method developed in Refs.~\cite{Ball:2011gg,Ball:2010gb} can be applied
to a native Monte Carlo set to quantify the impact at the PDF level
of a new experimental measurement.
%
The basic idea is that, starting from a sample of
$N_{\rm rep}$ MC replicas with each of them carrying equal weight,
the compatibility of each replica with the new
experimental dataset can be quantified by computing  a series of new weights
for each replica.
%
In this approach, the weight of the $k$-th replica is given by
\be
\label{eq:RWweights}
\omega_k = \frac{\lp \chi_k^2\rp^{(n-1)/2}e^{-\chi^2_k/2}/N_{\rm rep}}{\frac{1}{N_{\rm rep}}
\sum_{k=1}^{N_{\rm rep}}\lp \chi_k^2\rp^{(n-1)/2}e^{-\chi^2_k/2}} \, , \quad
k=1,\ldots,N_{\rm rep} \, ,
\ee
where $\chi^2_k$ is the goodness--of--fit estimator between the replica
$k$ and the new experimental measurement.
%
For instance, if for a given replica the agreement with the new
experiment is very poor, its $\chi^2_k$ will be large, and thus
the weight of this specific replica will be exponentially suppressed.
%
Note that by definition these new weights $\omega_k$
are appropriately normalized, and from the statistical point of view, they
can be interpreted as the probability of the replicas $f_k$, given
the $\chi^2_k$ for the new experimental measurement.
%
The validity of the Bayesian Monte Carlo method
 has been explicitly demonstrated by comparing
the reweighted results with those of a direct refit, finding
good agreement in all cases.


One of the limitations of the Bayesian reweighting method is that
it entails a given loss of information as compared to the initial prior,
because some of the original $N_{\rm rep}$ MC replicas will carry
a very small weight, meaning that they have been effectively discarded.
%
One suitable estimator to quantify this efficiency loss is the so-called
Shannon entropy, which allows  the effective number
of replicas left out after the reweighting to be evaluated.
%
Following standard information theory, the Shannon entropy of the reweighted
set, or more precisely, the effective number of replicas, is given by
\be
N_{\rm eff}\equiv \exp \lc \frac{1}{N_{\rm rep}}\sum_{k=1}^{N_{\rm rep}}
\omega_k \ln \lp N_{\rm rep}/\omega_k\rp\rc \, ,
\ee
where by construction, $0\le N_{\rm eff}\le N_{\rm rep}$.
%
The interpretation of this effective number of replicas is that
a reweighted PDF set carries the same amount of information
as a direct refit based on $N_{\rm eff}$ replicas.
%
Clearly, the smaller $N_{\rm eff}$ is, the more the new dataset constrains the
PDFs, but on the other hand if $N_{\rm eff}$ becomes
small enough, the reweighting method loses validity and a full
refit becomes necessary.
%
The Bayesian reweighting approach has been applied in a variety of 
PDF studies, both
for proton (unpolarized and polarized) PDFs,
see for instance
Refs.~\cite{d'Enterria:2012yj,Gauld:2016kpd,Czakon:2013tha,
Watt:2013oha,Nocera:2014gqa},
and for nuclear
PDFs~\cite{Armesto:2015lrg,Armesto:2013kqa,Paukkunen:2014zia,Kusina:2016fxy}. 

An advantage of the Bayesian reweighting method is that
it provides a way to estimate if the experimental uncertainties
have been either under or overestimated, assuming that theoretical
uncertainties are under control.
%
To achieve this, it is possible to rescale the total experimental
uncertainties of the data by a factor $\alpha$, and then
use inverse probability in order to evaluate the probability
density associated to the rescaling parameter $\alpha$, namely
\be
\label{eq:palpha}
\mathcal{P}\lp \alpha\rp \propto \frac{1}{\alpha}\sum_{k=1}^{N_{\rm rep}}
\omega_k(\alpha) \, ,
\ee
where the weights $\omega_k(\alpha)$ are computed using
Eq.~(\ref{eq:RWweights}) but replacing $\chi^2_k$ by $\chi^2_k/\alpha^2$,
and therefore represent the probability of $f_k$ given the new data
with rescaling error.
%
If this probability density Eq.~(\ref{eq:palpha}) peaks far above (below)
one, then this suggest that the uncertainties in the data have been
under (over) estimated, providing a useful handle to assess the
compatibility of a new measurement with a prior PDF analysis.


We
note that some authors have advocated a different
functional form for the weights than
that of Eq.~(\ref{eq:RWweights}),
see for instance~\cite{Giele:1998gw,Giele:2001mr,Sato:2013ika}.
%
However, only the definition of Eq.~(\ref{eq:RWweights}) has been shown
to lead to equivalent results in comparison to a direct
refit, within the reweighting accuracy.
%
It is also worth mentioning that the derivation of Eq.~(\ref{eq:RWweights})
assumes a native Monte Carlo PDF set, and that it does
not necessarily apply to MC sets that are obtained from native Hessian
sets using the conversion method of~\cite{Watt:2012tq},
see for example the discussions in~\cite{Paukkunen:2014zia,Kusina:2016fxy}

In order to provide an illustrative example of the
Bayesian reweighting method,
in Fig.~\ref{fig:rw} we show the gluon PDF in the NNPDF3.0
closure tests~\cite{Ball:2014uwa},
     estimating the impact of the collider inclusive jet data
     and
     comparing the results of the Bayesian reweighting
     with those of a direct refit.
     %
     In this study, the prior was a set of $N_{\rm rep}=1000$ replicas
     obtained with NNPDF2.3-like dataset but without any
     collider inclusive jet production data included.
     %
     The pseudo-data were generated using the MSTW08 NLO set,
     though similar results were obtained using
     other priors.
     %
     We observe that there is good agreement between the
     approximate Bayesian reweighting method and the exact refit
     results.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[t]
\begin{center}
  \includegraphics[scale=0.48]{4-fitting/plots/rw.pdf}
   \caption{\small 
     Left plot: the impact of the Tevatron $W$ and $Z$ data
     on the MMHT2014 NLO fit, estimated by the Hessian
     profiling method of Ref.~\cite{Camarda:2015zba}.
  %
     Right plot: the gluon PDF in the NNPDF3.0 closure tests,
     quantifying the impact of the collider inclusive jet data,
     and comparing the results of the Bayesian reweighting
     with those of a direct refit. 
    \label{fig:rw}
  }
\end{center}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsubsection{Hessian profiling}

For a Hessian PDF set, the so-called
profiling technique provides a method to approximately
quantity the impact of a new experimental measurement.
%
This method is based on the minimization of a
 $\chi^2$ estimator that compares
the theoretical predictions obtained with a given input Hessian
PDF set with the new experimental measurements.
%
This estimator takes into account both the experimental
uncertainties and the effects from the PDF variations (as encoded
by the Hessian
eigenvectors) and is defined as follows:
\be
\label{eq:hessianchi2}
\chi^2\lp {\rm \beta_{exp}},{\rm \beta_{th}}\rp
=\frac{1}{\Delta_i^2}\sum_{i=1}^{N_{\rm dat}}\lp \sigma_i^{\rm exp}
+\sum_j\Gamma_{ij}^{\rm exp}\beta_{j,\rm exp}
 -\sigma_i^{\rm th}
 +\sum_k\Gamma_{ik}^{\rm th}\beta_{k,\rm exp} \rp^2
 +\sum_j \beta_{j,\rm exp}^2+\sum_k \beta_{k,\rm th}^2 \, ,
 \ee
 where $\beta_{j,\rm exp}$ are the nuisance parameters corresponding
 to the set of fully correlated experimental systematic
 uncertainties, and $\beta_{k,\rm th}$ are the nuisance parameters
 corresponding to the PDF Hessian eigenvectors.
 %
 $\Delta_i$ is the total experimental
 uncorrelated uncertainty, and $N_{\rm dat}$ is the number of data
 points of the measurement which is being added into the
 PDF fit.
 %
 Finally, the matrices $\Gamma_{ij}^{\rm exp}$ and
 $\Gamma_{ik}^{\rm th}$ encode the effects of the corresponding
 nuisance parameters on the experimental data and on the
 theory predictions, respectively.
 
 Upon minimization of the $\chi^2$ estimator Eq.~(\ref{eq:hessianchi2}),
 the corresponding values of the theoretical nuisance parameters,
 denoted by 
 $\beta_{k,\rm th}^{\rm min}$, can be interpreted as leading
 to PDFs that have been optimized (hence the name ``profiled'')
 to describe this new specific measurement.
 %
 Note also that in general the profiling will modify both
 the central value and the total PDF uncertainty.
 %
 For example, the new measurement might reduce the allowed
 range of variation of a given eigenvector, if the original
 variation leads to large values of the figure of merit
 Eq.~(\ref{eq:hessianchi2}).

As in the case of the Bayesian reweighting method described
in the previous section,
there are a number of limitations of the the Hessian profiling method
that limit the cases where it can be used to replace
a complete refit.
 %
 First of all, it assumes that the optimal PDF parametrization will
 not be modified by the addition of the new experiment.
 %
 It is well known that this condition does not necessarily holds,
 for instance  new experiments
 might require the use
 of more flexible input PDF parametrizations in order
 to achieve an optimal description, and this effect cannot
 be accounted for with the profiling method.
 %
 Secondly, the standard version of the Hessian profiling method
 assumes that the PDF uncertainties are defined by the
 $\Delta\chi^2=1$ criterion, which is generally not the case for global
 Hessian PDF fits, see Sect.~\ref{sec:pdfuncertainties}.
 %
 For this reason, the impact of the data as estimated
 by Hessian profiling will in general differ in comparison
to the result of a full refit.
 %
 However, this limitation can be eliminated by using a tolerance criterion
 that mimics the one used in the prior Hessian PDF set,
 see for example Ref.~\cite{Kusina:2016fxy}.

As an example of the applications of the profiling
method, in Fig.~\ref{fig:rw} we show 
the impact of the Tevatron $W$ and $Z$ data
     on the MMHT2014 NLO set from Ref.~\cite{Camarda:2015zba},
     estimated by Hessian
     profiling.
%
This comparison shows
that these measurements lead
     to a reduction of the PDF uncertainties in the down valence PDF
     $d_V(x,Q)$.
     %
     An important point to emphasise here is that this exercise was performed
     using completely public tools, in this case the
     MMHT2014 {\tt LHAPDF} grids, and the experimental
     information of the Tevatron  $W$ and $Z$ measurements,
     without any additional input from the authors
     of the original MMHT2014 analysis.
     %
    This is the main advantage of Hessian profiling, the same as that of
    Bayesian reweighting: extending the possibility of carrying
    out PDF studies without the need of first having to produce a complete
    baseline global PDF fit.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5

