Score test

The statistic

Let ${\displaystyle L}$ be the likelihood function which depends on a univariate parameter ${\displaystyle \theta }$ and let ${\displaystyle x}$ be the data. The score ${\displaystyle U(\theta )}$ is defined as

${\displaystyle U(\theta )={\frac {\partial \log L(\theta \mid x)}{\partial \theta }}.}$

The Fisher information is^[6]

${\displaystyle I(\theta )=-\operatorname {E} \left[\left.{\frac {\partial ^{2}}{\partial \theta ^{2}}}\log f(X;\theta )\,\right|\,\theta \right]\,,}$

where ƒ is the probability density.

The statistic to test ${\displaystyle {\mathcal {H}}_{0}:\theta =\theta _{0}}$ is ${\displaystyle S(\theta _{0})={\frac {U(\theta _{0})^{2}}{I(\theta _{0})}}}$

which has an asymptotic distribution of ${\displaystyle \chi _{1}^{2}}$, when ${\displaystyle {\mathcal {H}}_{0}}$ is true. While asymptotically identical, calculating the LM statistic using the outer-gradient-product estimator of the Fisher information matrix can lead to bias in small samples.^[7]

As most powerful test for small deviations

${\displaystyle \left({\frac {\partial \log L(\theta \mid x)}{\partial \theta }}\right)_{\theta =\theta _{0}}\geq C}$

where ${\displaystyle L}$ is the likelihood function, ${\displaystyle \theta _{0}}$ is the value of the parameter of interest under the null hypothesis, and ${\displaystyle C}$ is a constant set depending on the size of the test desired (i.e. the probability of rejecting ${\displaystyle H_{0}}$ if ${\displaystyle H_{0}}$ is true; see Type I error).

The score test is the most powerful test for small deviations from ${\displaystyle H_{0}}$. To see this, consider testing ${\displaystyle \theta =\theta _{0}}$ versus ${\displaystyle \theta =\theta _{0}+h}$. By the Neyman–Pearson lemma, the most powerful test has the form

${\displaystyle {\frac {L(\theta _{0}+h\mid x)}{L(\theta _{0}\mid x)}}\geq K;}$

Taking the log of both sides yields

${\displaystyle \log L(\theta _{0}+h\mid x)-\log L(\theta _{0}\mid x)\geq \log K.}$

The score test follows making the substitution (by Taylor series expansion)

${\displaystyle \log L(\theta _{0}+h\mid x)\approx \log L(\theta _{0}\mid x)+h\times \left({\frac {\partial \log L(\theta \mid x)}{\partial \theta }}\right)_{\theta =\theta _{0}}}$

and identifying the ${\displaystyle C}$ above with ${\displaystyle \log(K)}$.

Relationship with other hypothesis tests

If the null hypothesis is true, the likelihood ratio test, the Wald test, and the score test are asymptotically equivalent tests of hypotheses.^[8][9] When testing nested models, the statistics for each test then converge to a Chi-squared distribution with degrees of freedom equal to the difference in degrees of freedom in the two models. If the null hypothesis is not true, however, the statistics converge to a noncentral chi-squared distribution with possibly different noncentrality parameters.