Examples Generalized linear model

1 examples

1.1 general linear models
1.2 linear regression
1.3 binary data

1.3.1 logit link function
1.3.2 probit link function popular choice of inverse cumulative distribution function
1.3.3 complementary log-log (cloglog)
1.3.4 identity link
1.3.5 variance function


1.4 multinomial regression

1.4.1 ordered response
1.4.2 unordered response


1.5 count data

examples
general linear models

a possible point of confusion has distinction between generalized linear models , general linear model, 2 broad statistical models. general linear model may viewed special case of generalized linear model identity link , responses distributed. exact results of interest obtained general linear model, general linear model has undergone longer historical development. results generalized linear model non-identity link asymptotic (tending work large samples).

linear regression

a simple, important example of generalized linear model (also example of general linear model) linear regression. in linear regression, use of least-squares estimator justified gauss–markov theorem, not assume distribution normal.

from perspective of generalized linear models, however, useful suppose distribution function normal distribution constant variance , link function identity, canonical link if variance known.

for normal distribution, generalized linear model has closed form expression maximum-likelihood estimates, convenient. other glms lack closed form estimates.

binary data

when response data, y, binary (taking on values 0 , 1), distribution function chosen bernoulli distribution , interpretation of μi probability, p, of yi taking on value one.

there several popular link functions binomial functions.

logit link function

the typical link function canonical logit link:

g
(
p
)
=
ln
⁡

(


p

1
−
p



)

.


{\displaystyle g(p)=\ln \left({p \over 1-p}\right).}

glms setup logistic regression models (or logit models).

probit link function popular choice of inverse cumulative distribution function

alternatively, inverse of continuous cumulative distribution function (cdf) can used link since cdf s range



[
0
,
1
]


{\displaystyle [0,1]}

, range of binomial mean. normal cdf



Φ


{\displaystyle \phi }

popular choice , yields probit model. link is

g
(
p
)
=

Φ

−
1


(
p
)
.




{\displaystyle g(p)=\phi ^{-1}(p).\,\!}

the reason use of probit model constant scaling of input variable normal cdf (which can absorbed through equivalent scaling of of parameters) yields function practically identical logit function, probit models more tractable in situations logit models. (in bayesian setting in distributed prior distributions placed on parameters, relationship between normal priors , normal cdf link function means probit model can computed using gibbs sampling, while logit model cannot.)

complementary log-log (cloglog)

the complementary log-log function may used:

g
(
p
)
=
log
⁡
(
−
log
⁡
(
1
−
p
)
)
.


{\displaystyle g(p)=\operatorname {log} (-\operatorname {log} (1-p)).}

this link function asymmetric , produce different results logit , probit link functions. cloglog model corresponds applications observe either 0 events (e.g., defects) or 1 or more, number of events assumed follow poisson distribution. poisson assumption means that

pr
⁡
(
0
)
=
exp
⁡
(
−
μ
)


{\displaystyle \operatorname {pr} (0)=\operatorname {exp} (-\mu )}

,

where μ positive number denoting inverse of expected number of events. if p represents proportion of observations @ least 1 event, complement

(
1
−
p
)
=

p
r
(
0
)

=
exp
⁡
(
−
μ
)


{\displaystyle (1-p)=\operatorname {pr(0)} =\operatorname {exp} (-\mu )}

,

and then

(
−
log
⁡
(
1
−
p
)
)
=
μ


{\displaystyle (-\operatorname {log} (1-p))=\mu }

.

a linear model requires response variable take values on entire real line. since μ must positive, can enforce taking logarithm, , letting log(μ) linear model. produces cloglog transformation

log
⁡
(
−
log
⁡
(
1
−
p
)
)
=
log
⁡
(
μ
)
.


{\displaystyle \log(-\log(1-p))=\log(\mu ).}



identity link

the identity link g(p) = p used binomial data yield linear probability model. however, identity link can predict nonsense probabilities less 0 or greater one. can avoided using transformation cloglog, probit or logit (or inverse cumulative distribution function). primary merit of identity link can estimated using linear math—and other standard link functions approximately linear matching identity link near p = 0.5.

variance function

the variance function quasibinomial data is:

var
⁡
(

y

i


)
=
τ

μ

i


(
1
−

μ

i


)




{\displaystyle \operatorname {var} (y_{i})=\tau \mu _{i}(1-\mu _{i})\,\!}

where dispersion parameter τ 1 binomial distribution. indeed, standard binomial likelihood omits τ. when present, model called quasibinomial , , modified likelihood called quasi-likelihood, since not likelihood corresponding real probability distribution. if τ exceeds 1, model said exhibit overdispersion.

multinomial regression

the binomial case may extended allow multinomial distribution response (also, generalized linear model counts, constrained total). there 2 ways in done:

ordered response

if response variable ordinal measurement, 1 may fit model function of form:

g
(

μ

m


)
=

η

m


=

β

0


+

x

1



β

1


+
⋯
+

x

p



β

p


+

γ

2


+
⋯
+

γ

m


=

η

1


+

γ

2


+
⋯
+

γ

m





{\displaystyle g(\mu _{m})=\eta _{m}=\beta _{0}+x_{1}\beta _{1}+\cdots +x_{p}\beta _{p}+\gamma _{2}+\cdots +\gamma _{m}=\eta _{1}+\gamma _{2}+\cdots +\gamma _{m}\,}

 




μ

m


=
p
⁡
(
y
≤
m
)



{\displaystyle \mu _{m}=\operatorname {p} (y\leq m)\,}

.

for m > 2. different links g lead proportional odds models or ordered probit models.

unordered response

if response variable nominal measurement, or data not satisfy assumptions of ordered model, 1 may fit model of following form:

g
(

μ

m


)
=

η

m


=

β

m
,
0


+

x

1



β

m
,
1


+
⋯
+

x

p



β

m
,
p



 where 


μ

m


=

p

(
y
=
m
∣
y
∈
{
1
,
m
}
)
.



{\displaystyle g(\mu _{m})=\eta _{m}=\beta _{m,0}+x_{1}\beta _{m,1}+\cdots +x_{p}\beta _{m,p}{\text{ }}\mu _{m}=\mathrm {p} (y=m\mid y\in \{1,m\}).\,}

for m > 2. different links g lead multinomial logit or multinomial probit models. these more general ordered response models, , more parameters estimated.

count data

another example of generalized linear models includes poisson regression models count data using poisson distribution. link typically logarithm, canonical link.

the variance function proportional mean

var
⁡
(

y

i


)
=
τ

μ

i


,



{\displaystyle \operatorname {var} (y_{i})=\tau \mu _{i},\,}

where dispersion parameter τ typically fixed @ one. when not, resulting quasi-likelihood model described poisson overdispersion or quasipoisson.