Uncertainty Quantification

The FFA framework implements three methods for uncertainty quantification:

  1. Sample bootstrap
  2. Regula-Falsi profile likelihood (RFPL)
  3. Regula-Falsi generalized profile likelihood (RFGPL)

Sample Bootstrap

The sample bootstrap is a flexible method for uncertainty quantification that works with all probability models and parameter estimation methods. Let \(n\) be the size of the original dataset.

  1. Draw \(N_{\text{sim}}\) bootstrap samples of size \(n\) from the selected probability distribution.
  2. Fit a probability distribution to each bootstrap sample using the same model selection method and parameter estimation method that was used to generate the original distribution.
  3. Compute the quantiles for each of the bootstrapped distributions.
  4. Generate confidence intervals using the mean and variance of the bootstrapped quantiles .

Handling Non-Stationarity

If the selected probability distribution is non-stationary, the quantiles for the bootstrapped distributions change in time. Therefore, the confidence intervals vary with time as well. Luckily, it is computationally inexpensive to determine the set of confidence intervals from the bootstrap distributions. Therefore, the FFA framework will report the confidence intervals for all years in the dataset by default when using the sample bootstrap quantification method.

Regula-Falsi Profile Likelihood (RFPL)

Consider a statistical model with parameters \((\theta, \psi_{1}, \dots, \psi_{n})\). The Profile Likelihood for the scalar parameter \(\theta\) and vector of nuisance parameters \(\psi\) is defined as:

\[ \ell_{p}(\theta) = \max_{\psi } \ell(\theta , \psi) \]

Let \(\hat{\theta}\) be MLE of \(\theta\). To find a confidence interval with significance \(1-\alpha\), we find the two solutions to the following equation (where \(\chi_{1;1-\alpha}^2\) is the \(1-\alpha\) quantile of the Chi-squared distribution):

\[ 2[\ell_{p}(\hat{\theta }) - \ell_{p}(\theta )] = \chi_{1;1-\alpha }^2 \]

This is equivalent to finding the two points \(\theta_{L} < \hat{\theta} < \theta_{U}\) such that the profile log-likelihood has dropped by \(\chi _{1;1-\alpha }^2 / 2\). To find \(\theta_{L}\) and \(\theta_{U}\) we find the roots of \(f(\theta)\) using a secant-based algorithm.

\[ f(\theta) = \ell_{p}(\theta) - \left[\ell_{p}(\hat{\theta}) - \frac{\chi_{1;1-\alpha }^2}{2}\right] \]

In the FFA framework, we compute the profile likelihood of each quantile \(y\) by reparameterizing the location parameter \(\mu\). Let \(q(p, \mu, \psi)\) be a function that takes an exceedance probability \(p\), location parameter \(\mu\) and nuisance parameters \(\psi\) and returns a quantile \(y\). All quantile functions satisfy:

\[ y = q(p, \mu, \psi) = \mu + q(p, 0, \psi) \]

Therefore, we can define \(\mu\) as a function of \((p, y, \psi)\) as shown below:

\[ \mu = y - q(p, 0, \psi) \]

We use this relationship to find the profile likelihood \(\ell_{p}(y)\) by evaluating \(\mu(p, y, \psi)\) and substituting it into the log-likelihood functions listed here.

Initialization Algorithm

Before we can find the roots of \(f\), we need to identify initial values for the regula-falsi algorithm:

  • Let \(a_{0}\) be a number such that \(a_{0} < y\) and \(f(a_{0}) < 0\).
  • Let \(b_{0}\) be a number such that \(b_{0} > y\) and \(f(b_{0}) < 0\).

To find \(a_{0}\), start by computing \(f(a^{*})\) for \(a^{*} = 0.95y\). If \(f(a^{*}) < 0\), then assign \(a_{0} = a^{*}\). Otherwise, update \(a^{*}\) to \(0.95a^{*}\) until \(f(a^{*}) < 0\). To find \(b_{0}\), we use a similar process. However, instead of iteratively revising \(b^{*}\) down, we revise it up to \(1.05b^{*}\).

Iteration Algorithm

At iteration \(i\), compute the following:

\[ c_{i} = \frac{a_{i-1}f(b_{i-1}) - b_{i-1}f(a_{i-1})}{f(b_{i-1}) - f(a_{i-1})} \]

Evaluate \(\ell_{p}(c_{i})\) by maximizing over the nuisance parameters \(\psi\), then find \(f(c_{i})\).

If \(|f(c_{i})| < \epsilon\) (where \(\epsilon\) is small), then stop. \(c_{i}\) is the confidence interval bound.

Otherwise, assign \(a_{i} = c_{i}\) if \(f(c_{i}) < 0\) and \(b_{i} = c_{i}\) if \(f(c_{i}) > 0\) and continue to iteration \(i + 1\).

Handling Non-Stationarity

Under non-stationarity, the quantiles \(y(t)\) vary with time. Therefore, we must execute the RFPL algorithm individually for each timestamp of interest. This can be quite computationally expensive, so the FFA framework defaults to running the RFPL algorithm on the last year in the dataset.

Regula-Falsi Generalized Profile Likelihood (RFGPL)

The regula-falsi generalized profile likelihood (RFGPL) method performs the regula-falsi algorithm shown above on the GEV distributions with a \(\text{Beta}(p, q)\) prior for the shape parameter \(\kappa\). For more information about generalized parameter estimation, see here.

Handling Non-Stationarity

See the section for the RFPL method.