We turn to discuss next how the quality of the agreement between
experimental data and theoretical predictions can be quantified
within a PDF fit, and the associated issue of the minimization strategy
adopted to find the optimal set of PDF parameters starting from
a figure of merit, typically the $\chi^2$.

\subsubsection{Fit quality and $\chi^2$ definition}\label{sec:fitmeth.min.chi2def}
%
The quality of the agreement between experimental measurements
and the corresponding theoretical
predictions within a global fit is usually expressed in terms of the log--likelihood function, or $\chi^2$.
When the correlations between the experimental systematic errors are not available,
the $\chi^2$ as a function of the PDF parameters is given by
\begin{equation}
\chi^2(\{a\})=\sum_{k=1}^{N_{pt}}{1\over \sigma_k^2}\left(D_k-T_k(\{a\})\right)^2,
\end{equation}
where $N_{pt}$ is number of data points, and $\sigma_k$
are the total experimental errors, given by adding the statistical and systematic
errors in quadrature.
%
In this expression,
$T_k(\{a\})$ are theoretical predictions,
expressed in terms of the PDF parameters $\{a\}$, and
$D_k$ are the central values of the experimental measurement.

Modern experiments provide correlated sources of the
various systematic uncertainties, in addition to the 
statistical and uncorrelated systematics.
%
The simplest example is the luminosity error in collider experiments, which is fully correlated among all
the bins from the same dataset.
%
Typically, there are many other sources that are introduced in the process of any given analysis, such as the Jet Energy Scale in jet production
measurements.
%
In such cases, the $\chi^2$ adopts the following form~\cite{Pumplin:2002vw}
\begin{equation}\label{eq:chi2}
\chi^2(\{a\},\{\lambda\})=\sum_{k=1}^{N_{pt}}{1\over s_k^2}\left(D_k-T_k-
\sum_{\alpha=1}^{N_{\lambda}}\beta_{k,\alpha}\lambda_{\alpha}\right)^2
+\sum_{\alpha=1}^{N_{\lambda}}\lambda_{\alpha}^2,
\end{equation}
for $N_{\lambda}$ sources of correlated error. Here, $s_k$ represents the total uncorrelated error, which is constructed
by adding the statistical error and uncorrelated systematic errors in quadrature.  
Each source of correlated systematic error is described by a nuisance
parameter $\lambda_{\alpha}$, with the error $\beta_{i,\alpha}$ correlated
among all data points.
%
Thus the induced systematic shift
to the experimental measurement is $\sum_{\alpha}\beta_{k,\alpha}\lambda_{\alpha}$.
The second sum on right side of Eq.~(\ref{eq:chi2}) includes the penalty terms
to the $\chi^2$, assuming standard Gaussian
distributions for the nuisance parameters.

In global PDF analyses we are more interested on the PDF parameters than
on the specific values that these nuisance parameters take.
%
Therefore, for any given set $\{a\}$ we can first
minimise the $\chi^2$ with respect to the nuisance parameters
$\lambda_\alpha$ to give the profiled log-likelihood
function $\chi^2(\{a\})\equiv\chi^2(\{a\},\{\hat \lambda\})$. 
%
While na\"{i}vely we might worry that this would be a computationally intensive exercise, the simple quadratic dependence of the $\chi^2$  on the $\lambda_{\alpha}$ allows the profiled nuisance parameter $\hat \lambda_{\alpha}$ to be solved for analytically, assuming purely Gaussian errors.  Explicitly, we have
\begin{equation}
\hat \lambda_{\alpha}=\sum_{i=1}^{N_{pt}}\frac{\left(D_i-T_i\right)}{s_i}
\sum_{\delta=1}^{N_{\lambda}}A^{-1}_{\alpha \delta}
\frac{\beta_{i,\delta}}{s_i},
\end{equation}
with
\begin{equation}
A_{\alpha\beta}=\delta_{\alpha\beta}+\sum_{k=1}^{N_{pt}}
\frac{\beta_{k,\alpha}\beta_{k,\beta}}{s_k^2}.
\end{equation}
By substituting $\hat \lambda_{\alpha}$ into the $\chi^2$ we
obtain the profiled $\chi^2$ as a function of the PDF parameters,
\begin{equation}
\chi^2(\{a\})=\sum_{i,j=1}^{N_{pt}}(T_i-D_i)({\rm cov}^{-1})_{ij}(T_j-D_j),
\end{equation}
with the experimental covariance matrix and its inverse given by
\begin{equation}
({\rm cov})_{ij}\equiv s_i^2\delta_{ij}+\sum_{\alpha=1}^{N_{\lambda}}\beta_{i,\alpha}
\beta_{j,\alpha}\, ,\,\,\quad ({\rm cov}^{-1})_{ij}=\frac{\delta_{ij}}{s_i^2}
-\sum_{\alpha,\beta=1}^{N_{\lambda}}
\frac{\beta_{i,\alpha}}{s_i^2}A^{-1}_{\alpha\beta}\frac{\beta_{j,\beta}}{s_j^2}.
\end{equation}
Thus, the profiled $\chi^2$ is fully determined by the covariance matrix,
which is  itself
constructed analytically in terms of the experimental statistical and systematic errors.
%
In certain circumstances, for example in the case of some
of the CMS and LHCb measurements,
the experiments publish the covariance matrix directly, instead of the full breakdown of the experimental systematics, which is less advantageous
from the PDF fitting point of view.

One final subtlety concerning the construction of the covariance matrix
arises due to the fact that experimental systematic errors are usually presented
as relative errors $\sigma_{i,\alpha}$ with respect to the data, that is
\begin{equation}
({\rm cov})_{ij}=s_i^2\delta_{ij}+\left(\sum_{\alpha=1}^{N_c}\sigma_{i,\alpha}^{(a)}
\sigma_{j,\alpha}^{(a)}+\sum_{\beta=1}^{N_{L}}\sigma_{i,\beta}^{(m)}\sigma_{j,\beta}^{(m)}
\right)D_iD_j.
\end{equation}
%
Here, we have further separated these sources into $N_c$ additive 
and $N_L$ multiplicative errors; in the former case this counts those errors that are absolute in size, while in the latter those sources (such as the luminosity) which genuinely correspond to a relative uncertainty on the data. 
%
These have quite different statistical interpretations, and indeed it is known that the above experimental
definition of the covariance matrix will result in a D'Agostini bias of the
multiplicative errors~\cite{Ball:2009qv} when used in a PDF fit.

To obtain unbiased results,
one should use the `$t_0$' definition of the covariance matrix~\cite{Ball:2009qv},  given by   
\begin{equation}
({\rm cov})_{ij}=s_i^2\delta_{ij}+\left(\sum_{\alpha=1}^{N_c}\sigma_{i,\alpha}^{(a)}
\sigma_{j,\alpha}^{(a)}D_iD_j+\sum_{\alpha=1}^{N_{L}}\sigma_{i,\beta}^{(n)}
\sigma_{j,\beta}^{(m)}T^{(0)}_iT^{(0)}_j\right).
\end{equation}
That is, we should rescale the multiplicative errors not by the data but by the theory prediction $T^0_i$,
from a previous iteration of the $\chi^2$ minimization.
%
An alternative prescription is the `{\it $t$}' definition~\cite{Ball:2012wy}, where the multiplicative errors are rescaled by the same theoretical prediction
as in the comparison to the data,
\begin{equation}
({\rm cov})_{ij}=s_i^2\delta_{ij}+\left(\sum_{\alpha=1}^{N_c}\sigma_{i,\alpha}^{(a)}
\sigma_{j,\alpha}^{(a)}D_iD_j+\sum_{\alpha=1}^{N_{L}}\sigma_{i,\beta}^{(m)}
\sigma_{j,\beta}^{(m)}T_iT_j\right),
\end{equation}
or the {\it extended-$t_0$} and {\it extended-$t$}
definitions, where  both the additive and multiplicative errors are rescaled by the
corresponding theory.
%
A detailed discussion of the various possible prescriptions
can be found in~\cite{Ball:2012wy,Andersen:2014efa}.    

\subsubsection{Minimization strategies}
\label{sec:fitmeth.min.chi2min}
%
The central PDF fits are determined by finding the global minimum of the log-likelihood function $\chi^2(\{a\})$. For PDF sets with a moderate number of free parameters, say less than $N_{\rm par}\simeq 40$,
numerical gradient--based algorithms are typically used. As a simple example, in Newton's method, the trial solution for the global minimum is given by
\be
a^{\rm trial}_i=a^0_i-\sum_{j=1}^{n_{\rm par}}H^{-1}_{ij}d_j\;,
\ee
for the $i$th PDF parameter. Here $a^0$ is an arbitrary starting point in the PDF parameter space, $d$ is the gradient and $H^{-1}$ is the inverse of the Hessian matrix (defined in Sect.~\ref{sec:pdfuncertainties.hess}) at the same point.
%
This solution is exact assuming a purely quadratic shape for the $\chi^2$, although in practice it can deviate significantly from this when it is far away from the global minimum. The above solution is therefore typically applied iteratively until the desired degree of convergence is achieved. However, the method will fail if the Hessian matrix $H$ is not positive--definite, and can suffer from numerical instabilities.
%
Various quasi--Newton
methods have been proposed to overcome these complexities in real applications, such as the Levenberg-Marquardt method used in MSTW/MMHT analyses~\cite{Martin:2009iq},
which is based on
a dynamically determined combination of Newton's method and the steepest decent method. 

In gradient-based methods, the gradient and Hessian matrix must be calculated numerically by
means of finite differences.
%
Another class of widely used gradient based algorithms are the variable metric methods (VMM), where it is not necessary to calculate the
Hessian matrix numerically.
%
Instead, the matrix $H$
is updated iteratively based only on information of the gradients. VMM
is the default algorithm in the MINUIT package~\cite{James:1975dr} and is used
in CTEQ-TEA analyses~\cite{Pumplin:2000vx}.   

As the number of free parameters is increased, the above methods will begin to suffer from numerical instabilities and issues with local minima.
%
For the NNPDF analysis, where the typical number of parameters is an order of magnitude higher than in other sets, a genetic algorithm~\cite{tau,GonzalezGarcia:2006ay,quevedo}
is the appropriate choice, as demonstrated in~\cite{Ball:2010de}.
%
The basic idea is to start from an ensemble of arbitrarily chosen samples 
of the PDF parameters. Random mutations with possible crossing--overs
are then applied to generate a larger group of new samples. Those
candidates predicting a lower $\chi^2$ are then selected to form a
new ensemble with the same size.
%
This procedure is then
iterated until a suitable convergence criterion is met, while care is taken to
prevent over-fitting.    
