Confronting Predictions with Experimental Limits¶
Once the relevant signal cross sections (or theory predictions) have been computed for the input model, these must be compared to the respective upper limits. The upper limits for the signal are stored in the SModelS Database and depend on the type of Experimental Result: UL-type or EM-type.
In the case of a UL-type result, the theory predictions typically consist of a list of signal cross sections (one for each cluster) for the single data set (see Theory Predictions for Upper Limit Results for more details). Each theory prediction must then be compared to its corresponding upper limit. This limit is simply the cross section upper limit provided by the experimental publication or conference note and is extracted from the corresponding UL map (see UL-type results).
For EM-type results there is a single cluster for each data set (or signal region), and hence a single signal cross section value. This value must be compared to the upper limit for the corresponding signal region. This upper limit is easily computed using the number of observed and expected events for the data set and their uncertainties and is typically stored in the Database. Since most EM-type results have several signal regions (data sets), there will be one theory prediction/upper limit for each data set. By default SModelS keeps only the best data set, i.e. the one with the largest ratio \(\mbox{(theory prediction)}/\mbox{(expected limit)}\). Thus each EM-type result will have a single theory prediction/upper limit, corresponding to the best data set (based on the expected limit). If the user wants to have access to all the data sets, the default behavior can be disabled by setting useBestDataset=False in theoryPredictionsFor (see Example.py).
The procedure described above can be applied to all the Experimental Results in the database, resulting in a list of theory predictions and upper limits for each Experimental Result. A model can then be considered excluded by the experimental results if, for one or more predictions, we have theory prediction \(>\) upper limit [*].
- The upper limits for a given UL-type result or EM-type result can be obtained using the getUpperLimitFor method
Likelihood Computation¶
For EM-type results a \(\chi^2\)-value can also be computed (in addition to the upper limits above). The \(\chi^2\) is computed from the likelihood using:
where \(L(n_{\mathrm{signal}})\) is the likelihood for a given number of signal events. The likelihood is computed using a simple Poisson convoluted with a Gaussian (for the background and signal uncertainties) as a function of the number of observed events (\(n_{\mathrm{obs}}\)), the number of expected background events (\(n_{b}\)) and its error (\(\delta_{b}\)) and the number of signal events (\(n_{\mathrm{signal}}\)) and its error (\(\delta_{s})\)). While \(n_{\mathrm{obs}}\), \(n_{b}\) and \(\delta_{b}\) are directly extracted from the data set, \(n_{\mathrm{signal}}\) is obtained from the Theory Predictions calculation and \(\delta_{s} = 20\%~\cdot n_{\mathrm{signal}}\) by default.
- The \(\chi^2\) for a given EM-type result is computed using the chi2 method
[*] | The statistical significance of the exclusion statement is difficult to quantify exactly, since the model is being tested by a large number of results simultaneously. |