Lecture 4: Bootstrap
Bootstrap
Bias Correction with Bootstrap Bootstrap Confidence Intervals Bootstrap for Regression
- Sampling distribution of θˆis hard to derive.
)Or, we just don’t feel like taking the time to.
- Our sample size is not tiny but is too small for “asymptopia”.
Nonparametric bootstrap: Let Y= [Y, . . . , Y] iidF
- ) (some unknown CDF).
1 n ∼Y
-
- Fˆ(y)
⇒ = The distribution of the sample approximates the distribution of the population as nincreases.
⇒ = Sampling from a sufficiently large sample approximates sampling from the population.
Parametricbootstrap:► Ifanestimatorθˆnisconsistentandthemodeliscorrect,
p(·|θˆn)approximatesp(·|θ).
=⇒Samplingfromp(·|θˆn)approximatessamplingfromthepopulation.
Notation: Y s = [Y s , . . . , Y s ] the bth “resample” with replacement from observed
b b1 bn ˆ
ˆ vector Y or p(·|θn)
≡ θ(FY) θ the true parameter value or some population quantity function of
FY
Bootstrap
Bias Correction with Bootstrap Bootstrap Confidence Intervals Bootstrap for Regression
Let Y1, . . . , Y
iid
fn ∼Y
(y; θ) = θe−θy, for y> 0 and θ > 0.
Σn
l(θ) = (log θ − θYi) = nlog θ − θnY¯
i =1
Σ► But,
n i=1
Yi∼ Ga(a= n, b= θ)
θˆ= (Y¯)−1
- Y¯∼ Ga(a= n, b= nθ)
- n 1 E(θˆ) = E(1/Y¯) = EInvGa(a= n, b= nθ) = nθ > θ
Also, by Jensen’s Inequality, −
- E(Y¯) = 1/θ (a convex function)
∴ E(θˆ) = E(1/Y¯) > 1/ E(Y¯) = θ.
Recall,
FY population from which Y are drawn
θˆ(FY) population quantity
θˆn(Y) some estimator of θˆ(FY)
biasFY[θˆn(Y)] = EFY[θˆn(Y)] − θˆ(FY)
- An estimator θˆn(Y) is biasedif biasFY[θˆn(Y)] ƒ= 0
- Note that Wakefield (2013, Sec. 2.7.1) implicitly defines it the other way, as
θˆ(FY) − EFY[θˆn(Y)].
- Jensen’s inequality: If it happens that E(Y) = g(θ) for some g, and θˆ= g−1(Y¯), and g(y) is concave or convex, θˆwill be biased for θ.
0.5
n <- 20 # Samplesize
theta.true <- 1 #Exponentialdistributiontruerateparameter
S <- 1000 # Number of replications for the simulation
B <- 999 #Bootstrapresamplesize
#Afunctiontogenerateadataset
mk.data <- function(n, theta=theta.true) rexp(n,theta)
# MLE
theta.mle <- function(y) 1/mean(y)
# MLE sampling distribution
theta.mle.dist <- replicate(S, y.obs <- mk.data(n) theta.mle(y.obs)
)
mean(theta.mle.dist) # Mean of the sampling distribution
## [1] 1.047768
mean(theta.mle.dist)-theta.true #Bias
## [1] 0.04776839
1.0 1.5 2.0 * = mean
b
b
- Suppose we have a biased estimator:
biasFY[θˆn(Y)] = E[θˆn(Y)] − θˆ(FY) =ƒ 0.
) We don’t know either expectation.
- Idea:treatFˆnasF.
- Ys∼Fˆn,soθˆ(Ys)willbebiasedrelativetoθˆ(Y)≡θˆ(Fˆn)byapproximatelythe
b b
same amount:
biasˆn[θˆn(Ys)] = Eˆn[θˆn(Ys)] − θˆn(Y) ≈ EF[θˆn(Y)] − θˆ(FY) = biasF[θˆn(Y)]
F b F b Y
B
B
b
b
cancel some of the bias:
B
b=1
b
b=1
≈1Σθˆn(Ys)−θˆn(Y)
=⇒Subtractingoffθˆn(Ys)−θˆn(Y)=1ΣBθˆn(Ys)−θˆn(Y)fromθˆn(Y)can
θˆn,u(Y)=θˆn(Y)−.θˆn(Ys)−θˆn(Y)Σ =2θˆn(Y)−θˆn(Ys)
# Nonparamametric bootstrap sampling
myboot.np <- function(B, y.obs, theta.f)
replicate(B,
y.b <- sample(y.obs,replace=TRUE) theta.f(y.b)
)
* = mean
# Sampling distribution of the mean of the bootstrap samples
2.0 theta.boot.dist <- replicate(S, y.obs <- mk.data(n)
mean(myboot.np(B, y.obs, theta.mle))
)
1.5mean(theta.boot.dist)-theta.true #Biasedbyabouttwiceasmuch
## [1] 0.1177986
# Debiased estimate and simulation
1.0 theta.bc <- function(theta.obs, theta.boot) theta.obs – (mean(theta.boot) – theta.obs)
0.5
theta.bc.dist <- replicate(S, y.obs <- mk.data(n)
theta.obs <- theta.mle(y.obs)
theta.boot <- myboot.np(B, y.obs, theta.mle)
theta.bc(theta.obs,theta.boot)
)
mean(theta.bc.dist)-theta.true #Muchlessbiased.
## [1] -0.001975362
MLE
Bootstrap
Bias Correction with Bootstrap Bootstrap Confidence Intervals Bootstrap for Regression
Bootstrap
Bias Correction with Bootstrap Bootstrap Confidence Intervals
Basic approaches
Refined approaches Demonstration
Bootstrap for Regression
Wakefield (2013, Sec. 2.7.1); Carpenter and Bithell (2000)
Given
Ys, . . . , Ys
B resamples of the dataset
1 B
b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest
b θˆqsqth quantile of θˆ(Ys)s,
α = 1 − CL
.θ − (θ ˆ ˆs
1−α/2
− θˆ), θˆ− (θˆs
- θˆ)Σ
α/2
− − − Assumes that distribution of θˆsθˆis the same as the distribution of θˆθ.
0.975 Intuition: 97.5% of the time, θˆwill overestimate θ by less than θˆs
− θˆ,
0.975 so θ will be less than θˆ− (θˆs− θˆ) only 2.5% of the time.
− Often has poor coverage.
Y s, . . . , Y s B resamples of the dataset
1 B
b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest
b θˆqsqth quantile of θˆ(Ys)s,
α = 1 − CL
θˆ(Y) − .θˆ(Ys) − θˆ(Y)Σ ± zs
b
B
b=1
b
√v^arF(θˆ),
b
where θˆ(Ys) = 1ΣBθˆ(Ys) and
1−α/2
^
F θ
B
b=1
θ
b
− B
b=1 θ
b )
. var (ˆ) = 1ΣB .ˆ(Ys) 1ΣB ˆ(Y sΣ2
− Assumes sampling distribution of θˆis symmetric and not too weird.
Intuition: Use bootstrap to debias and estimate the variance.
+ Has good coverage if it is.
^ + Can be useful if var(θˆ) is hard to get analytically.
Y s, . . . , Y s B resamples of the dataset
1 B
b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest
b θˆqsqth quantile of θˆ(Ys)s,
α = 1 − CL
+ Invariant to transformation.
.θˆsα/2, θˆsΣ
1−α/2 − Effectively assumes that there exists a monotone transformation g (θ) s.t.
g(θˆ) ∼ N(g(θ), σ2)
g(θˆs) ∼ N(g(θˆ), σ2)
Intuition: If it exists we could get a 1 − α CI by taking
g−1g(θˆ) ± zsσ, but g(θˆ) + zsσ ≡ g(θˆs
), so we
1−α/2 ˆs
1−α/2
1−α/2
can just use quantiles of θ directly.
)Performs poorly if θˆdoesn’t behave that way (e.g., mean–variance relationship).
Bootstrap
Bias Correction with Bootstrap Bootstrap Confidence Intervals
Basic approaches
Refined approaches Demonstration
Bootstrap for Regression
Y s, . . . , Y s B resamples of the dataset
1 B
b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest
b θˆqsqth quantile of θˆ(Ys)s,
α = 1 − CL
- A refinement of pivotal interval
- √ Adjusts for difference between var(θˆs− θˆ) and var(θˆ− θ).
- Estimate σˆ ≈ varθ(θˆ).
- For each θˆs, estimate σˆs≈
.var ˆs (θˆ)
b
b b θb)E.g., run bootstrap for every θˆs.
)Very computationally intensive, unless a formula exists for σˆ.
- Calculate ts(θˆs) = (θˆs− θˆ)/σˆsfor b= 1..B, tsits qth quantile.
- 1−α/2 α/2 The CI is then .θˆ+ σˆts, θˆ− σˆtsΣ.
Y s, . . . , Y s B resamples of the dataset
1 B
b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest
b θˆqsqth quantile of θˆ(Ys)s,
α = 1 − CL
- A refinement of percentile interval
+ Adjusts for skewness in θˆand in how it changes with θ.
) Loosely, assumes that a transformation g (θ) exists s.t.
g(θˆ) ∼ N[g(θ) − b1 + ag(θ), 1 + ag(θ)2]
g(θˆs) ∼ N[g(θˆ) − b1 + ag(θˆ), 1 + ag(θˆ)2]
for a and b.
+ Works very well
− Can misbehave badly if α very small.
- Then,qu
= Φ ,b−
s α/2
z 1+a(zs
−b
−b)
,, ql = Φ ,b−
zs
1−α/2 1+a(zs
−b
−b)
,, and the CI is
ˆ)More complicated afor parametric.
l
u.θˆqs, θˆqsΣ.
α/2
1−α/2
Bootstrap
Bias Correction with Bootstrap Bootstrap Confidence Intervals
Basic approaches
Refined approaches Demonstration
Bootstrap for Regression
# Asymptotic CI using I(theta):
theta.asymp.CI <- function(n, theta.obs) theta.obs+qnorm(c(.025,.975))*sqrt(theta.obs^2/n)
# Bootstrap CI using normal approximation:
theta.boot.N.CI <- function(n, theta.obs, theta.boot) bias <- mean(theta.boot) – theta.obs
theta.obs – bias + qnorm(c(.025,.975))*sd(theta.boot)
# Bootstrap CI using quantiles:
theta.boot.Q.CI <- function(n, theta.obs, theta.boot)
quantile(theta.boot,c(0.025,0.975))
# Bootstrap CI using BCa:
theta.boot.BCa.CI <- function(n, theta.obs, theta.boot) b <- qnorm(mean(theta.boot<theta.obs))
a <- – mean((theta.boot-mean(theta.boot))^3)/sd(theta.boot)^3 / 6 z.minus.b <- qnorm(c(.975,.025))-b
q <- pnorm(b-z.minus.b/(1+a*z.minus.b)) quantile(theta.boot, q)
rbind(Asymptotic=theta.asymp.CI(n, theta.obs), Normal=theta.boot.N.CI(n, theta.obs, theta.boot), Quantile=theta.boot.Q.CI(n, theta.obs, theta.boot), Basic=theta.boot.pivot.CI(n, theta.obs, theta.boot), BCa=theta.boot.BCa.CI(n, theta.obs, theta.boot))
## 2.5 97.5
## Asymptotic 0.5690201 1.456904
## Normal 0.5019225 1.430258
## Quantile 0.6873244 1.605447
## Basic 0.4204773 1.338600
## BCa 0.5725606 1.466644
- Automates bootstrapping and CI calculation
- For ordinary (default) bootstrap, “statistic” is a function of the original dataset and indices selecting the resample.
library(boot)
theta.Rboot <- boot(y.obs, #data
function(y.obs,i) theta.mle(y.obs[i]), B) #Statistic
boot.ci(theta.Rboot) # Get theCIs
## Warning in boot.ci(theta.Rboot): bootstrap variances needed for studentized intervals
## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
## Based on 999 bootstrap replicates ##
## CALL :
## boot.ci(boot.out = theta.Rboot) ##
## Intervals :
## Level Normal Basic
## 95 ( 0.447, 1.469 ) ( 0.357, 1.348 ) ##
## Level Percentile BCa
## 95 ( 0.678, 1.669 ) ( 0.644, 1.559 )
## Calculations and Intervals on Original Scale
- A coverage simulation examines whether a statistical method produces good confidence intervals under a variety of assumptions.
- Simulate Y from some known parameter θ.
- Calculate a CI for θ based on Y.
- How often does the CI actually contain θ? A 95% CI should contain it 95% of the time, etc..
S <- 1000 # Simulationsize
covers <- function(CI, truth) CI[1]<=truth && CI[2]>=truth
cover.sim <- function(mk.data.f, boot.f)
replicate(S,
y.obs <- mk.data.f(n)
theta.boot <- boot.f(B, y.obs, theta.mle) theta.obs <- theta.mle(y.obs)
c(Asymptotic=covers(theta.asymp.CI(n, theta.obs),theta.true),
`Normal`=covers(theta.boot.N.CI(n, theta.obs, theta.boot),theta.true),
`Quantile`=covers(theta.boot.Q.CI(n, theta.obs, theta.boot),theta.true),
`Basic`=covers(theta.boot.pivot.CI(n, theta.obs, theta.boot),theta.true),
`BCa`=covers(theta.boot.BCa.CI(n, theta.obs, theta.boot),theta.true))
)
| rowMeans(cover.sim(mk.data, myboot.np)) | ||||
| ## Asymptotic | Normal | Quantile | Basic | BCa |
| ## 0.953 | 0.939 | 0.893 | 0.891 | 0.910 |
theta.boot <- myboot.p(B, y.obs, theta.mle)
#ThisBCaOKinthiscase,butiffyingeneral rbind(Asymptotic=theta.asymp.CI(n, theta.obs), Normal=theta.boot.N.CI(n, theta.obs, theta.boot), Quantile=theta.boot.Q.CI(n, theta.obs, theta.boot), Basic=theta.boot.pivot.CI(n, theta.obs, theta.boot), BCa=theta.boot.BCa.CI(n, theta.obs, theta.boot))
## 2.5 97.5
## Asymptotic 0.5690201 1.456904
## Normal 0.4853627 1.449853
## Quantile 0.6847231 1.647648
## Basic 0.3782760 1.341201
## BCa 0.5562614 1.385008
| rowMeans(cover.sim(mk.data, myboot.p)) | ||||
| ## Asymptotic | Normal | Quantile | Basic | BCa |
| ## 0.952 | 0.963 | 0.936 | 0.918 | 0.942 |
- statistic is just a function of data.
- ran.gen is a function of original data and parameters passed via mle that generates a replication of the dataset.
- BCa is not as straightforward, because a’s estimation depends on model properties. (This workaround might be a bit iffy.)
theta.Rboot <- boot(y.obs, theta.mle, B, sim=”parametric”, ran.gen=function(d, p) mk.data(length(d), p),mle=theta.obs)
boot.ci(theta.Rboot, L=empinf(data=y.obs, stype=”i”, type=”jack”, statistic=function(y.obs,
## Warning in boot.ci(theta.Rboot, L = empinf(data = y.obs, stype = “i”, type = “jack”, : bootstrap variances needed for studentized intervals
## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
## Based on 999 bootstrap replicates ##
## CALL :
## boot.ci(boot.out = theta.Rboot, L = empinf(data = y.obs, stype = “i”, ## type = “jack”, statistic = function(y.obs, i) theta.mle(y.obs[i]))) ##
## Intervals :
## Level Normal Basic
## 95 ( 0.449, 1.469 ) ( 0.298, 1.326 ) ##
## Level Percentile BCa
## 95 ( 0.700, 1.728 ) ( 0.645, 1.523 )
## Calculations and Intervals on Original Scale ## Some BCa intervals may be unstable
- Now, suppose that we were wrong about our data: our distribution was, in fact, Ga(a= 4, b= 4), not exponential: same mean, but smaller SD.
mk.data2 <- function(n) rgamma(n,4,theta.true*4)
Nonparametric bootstrap:
| rowMeans(cover.sim(mk.data2, myboot.np)) | |||
| ## Asymptotic | Normal Quantile | Basic | BCa |
| ## 1.000 | 0.932 0.915 | 0.917 | 0.920 |
| Parametric bootstrap: | |||
| rowMeans(cover.sim(mk.data2, myboot.p)) | |||
| ## Asymptotic | Normal Quantile | Basic | BCa |
| ## 1.000 | 1.000 1.000 | 0.993 | 0.999 |
Bootstrap
Bias Correction with Bootstrap Bootstrap Confidence Intervals Bootstrap for Regression
Wakefield (2013, Sec. 2.7.2)
We know what to do with Yi s, but what about xi s?
- Resample cases ((Yi, xi))
+ Generally robust to, e.g. variance assumptions.
− Ignores the sampling design.
)If the sampling is blocked, stratified, etc., bootstrap resampling can be modified to reflect that.
- | − | Resample responses Yi xi (parametric) or residuals (Yi µi(β)) xi, β
) Residuals can also be resampled parametrically.
) If the model postulates a mean-variance relationship, one can resample studentised (divided by predicted variance) residuals.
+ Preserves sampling design.
+ Robust to distributional assumptions (if nonparametric).
− Not robust to mean-variance relationship assumptions.
- Nonparametric resampling of residuals would be problematic, since the variance increases with the mean.
- Parametric bootstrap produces results comparable to the likelihood standard errors, while resampling cases produces results comparable to the sandwich standard errors.
load(“Radon_Example.RData”)
#Afunctiontofitthemodelofinterest,usingdata,selectingrowsspecifiedbyinds(th
do.poisfit <- function(data, inds=seq_len(nrow(data)), coef=TRUE) data <- data[inds,] # Select only the specified rows
poisfit <- glm(Obs~offset(log(Exp))+radon,family=poisson, na.action=na.omit,data=data) ##Ifcoef=TRUE,extractthecoefficients.Otherwise,returnthefullfitobject. if(coef) coef(poisfit) elsepoisfit
# The MLE
poisfit <- do.poisfit(MNrad,coef=FALSE) theta.obs <- coef(poisfit)
n <- nrow(MNrad)
ms <- predict(poisfit, type=”response”)
MNrad.boot <- MNrad
theta.boot <- t(replicate(B,
MNrad.boot$Obs <- rpois(length(ms), ms) # Generate new cancercounts
do.poisfit(MNrad.boot)
))
cov(theta.boot)
## (Intercept) radon ## (Intercept) 0.0007113803 -1.349959e-04 ## radon -0.0001349959 2.855784e-05
vcov(poisfit)
## (Intercept) radon ## (Intercept) 0.0007186984 -1.370103e-04 ## radon -0.0001370103 2.913701e-05
lapply(1:2, function(i) rbind(Asymptotic=coef(poisfit)[i]+qnorm(c(.025,.975))*sqrt(diag(vco
Normal=theta.boot.N.CI(n, theta.obs[i], theta.boot[,i]), Quantile=theta.boot.Q.CI(n, theta.obs[i], theta.boot[,i]), Basic=theta.boot.pivot.CI(n, theta.obs[i], theta.boot[,i]), BCa=theta.boot.BCa.CI(n, theta.obs[i], theta.boot[,i])))
## [[1]]
## 2.5 97.5
## Asymptotic 0.1180851 0.2231727
## Normal 0.1184123 0.2229635
## Quantile 0.1201091 0.2229689
## Basic 0.1182889 0.2211487
## BCa 0.1219132 0.2237508
##
## [[2]]
## 2.5 97.5
## Asymptotic -0.04673742 -0.02557815
## Normal -0.04658703 -0.02563912
## Quantile -0.04676029 -0.02589079
## Basic -0.04642478 -0.02555528
## BCa -0.04681753 -0.02627902
- You have to “ask” for each variable at a time.
theta.boot <- boot(MNrad, statistic=do.poisfit, sim=”parametric”, R=B, ran.gen = function( boot.ci(theta.boot, index=2, L=empinf(data=MNrad, stype=”i”, type=”jack”, statistic=do.po
## Warning in boot.ci(theta.boot, index = 2, L = empinf(data = MNrad, stype = “i”,
: bootstrap variances needed for studentized intervals
## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
## Based on 999 bootstrap replicates ##
## CALL :
## boot.ci(boot.out = theta.boot, index = 2, L = empinf(data = MNrad, ## stype = “i”, type = “jack”, statistic = do.poisfit))
##
## Intervals :
## Level Normal Basic
## 95 (-0.0465, -0.0258 ) (-0.0467, -0.0259 ) ##
## Level Percentile BCa
## 95 (-0.0464, -0.0257 ) (-0.0458, -0.0245 )
## Calculations and Intervals on Original Scale
theta.boot <- t(replicate(B,
#Wewantwhole(Obs,Exp,x)triplets,soweneedtoselectrowsinthedataframe.
inds <- sample.int(nrow(MNrad),replace=TRUE) do.poisfit(MNrad,inds)
))
cov(theta.boot)
## (Intercept) radon ## (Intercept) 0.0021112536 -3.423497e-04 ## radon -0.0003423497 7.314646e-05
vcov(poisfit)
## (Intercept) radon ## (Intercept) 0.0007186984 -1.370103e-04 ## radon -0.0001370103 2.913701e-05
sandwich(poisfit)
## (Intercept) radon ## (Intercept) 0.0018419895 -2.985599e-04 ## radon -0.0002985599 6.438377e-05
lapply(1:2, function(i) rbind(Asymptotic=coef(poisfit)[i]+qnorm(c(.025,.975))*sqrt(diag(vco
Normal=theta.boot.N.CI(n, theta.obs[i], theta.boot[,i]), Quantile=theta.boot.Q.CI(n, theta.obs[i], theta.boot[,i]), Basic=theta.boot.pivot.CI(n, theta.obs[i], theta.boot[,i]), BCa=theta.boot.BCa.CI(n, theta.obs[i], theta.boot[,i])))
## [[1]]
## 2.5 97.5
## Asymptotic 0.11808508 0.2231727
## Normal 0.08484135 0.2649557
## Quantile 0.07699282 0.2497400
## Basic 0.09151778 0.2642649
## BCa 0.08353562 0.2594277
##
## [[2]]
## 2.5 97.5
## Asymptotic -0.04673742 -0.02557815
## Normal -0.05277069 -0.01924523
## Quantile -0.05347350 -0.02025274
## Basic -0.05206283 -0.01884207
## BCa -0.05344392 -0.02020988
theta.boot <- boot(MNrad, statistic=do.poisfit, sim=”ordinary”, R=B)
suppressWarnings(boot.ci(theta.boot, index=2))
## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
## Based on 999 bootstrap replicates ##
## CALL :
## boot.ci(boot.out = theta.boot, index = 2) ##
## Intervals :
## Level Normal Basic
## 95 (-0.0527, -0.0200 ) (-0.0535, -0.0200 ) ##
## Level Percentile BCa
## 95 (-0.0523, -0.0188 ) (-0.0523, -0.0184 )
## Calculations and Intervals on Original Scale
- Very robust and flexible, but tends to have slightly low coverage, particularly nonparametric.
- Must reflect the sampling process, e.g.,
Stratification: Resample from each stratum in accordance with its sample size. Serial (temporal) dependence: Use block bootstrap methods.
James Carpenter and John Bithell. Bootstrap confidence intervals: When, which, what? A practical guide for medical statisticians. Statistics in Medicine, 19: 1141–1164, 2000.
Anthony C. Davison and David V. Hinkley. Bootstrap Methods and their Application. Cambridge University Press, Cambridge, 1997. ISBN 0-521-57391-2. URL http://statwww.epfl.ch/davison/BMA/.
Jon Wakefield. BayesianandFrequentistRegressionMethods. Springer Series in Statistics. Springer New York, 2013. ISBN 978-1-4419-0925-1. doi:10.1007/978-1-4419-0925-1.
Let Xbe a r.v. that is not degenerate(can have multiple values), and let ϕ(x) be a smooth function of xin the support of X(i.e., possible values of X).
- ϕE(X) = Eϕ(X) if ϕ(x) is linear (i.e., ϕjj(x) = 0 in the support of X)
- X) ϕE(X) < Eϕ(X) if ϕ(x) is strictly convex (i.e., ϕjj(x) > 0 in the support of
- X) ϕE(X) > Eϕ(X) if ϕ(x) is strictly concave (i.e., ϕjj(x) < 0 in the support of
Similarly with sample means: if ϕ(x) ≡ 1Σn
ϕ(xi ), if not all xi s are the same:
n i=1
- ϕx¯ = ϕ(x) if ϕ(x) is linear
- ϕx¯ < ϕ(x) if ϕ(x) is strictly convex
- ϕx¯ > ϕ(x) if ϕ(x) is strictly concave
i=1
Qn Example: For xi> 0 not all the same, geometric mean (
xi)1/nis smaller than
n
i=1
1.
Let
ϕ(x)
=
log(x).
ϕjj(x)
=
−x−2<
0.
the arithmetic
mean 1 Σn
xi
:
- (Qnxi)1/n= exp[log(Qn
n
i=1
xi)1/n] = exp[ 1Σnlogxi].
- i=1 i=1 i=1 i=1 1 Σn logxi < log 1Σn
xi = logx¯ (Jensen’s Inequality)
- Σn exp(x) is strictly increasing, so
n
i=1x¯ = explog(x¯) > exp[ 1
logxi] = (Qn
xi)1/n.
Wakefield (2013, Ex. 2.9)
- − − − Consider an interval in the form of (θˆH−1(1 α/2), θˆH−1(α/2)), where
- −
H( ) is the CDF of the sampling distribution of θˆθ.
-
- −
H−1(q) + θ = F−1(q).
- Then,
Pr [θˆ− H−1(1 − α/2) < θ < θˆ− H−1(α/2)]
= Pr[−H−1(1 − α/2) − θ < −θˆ< −H−1(α/2) − θ]
= Pr[H−1(α/2) + θ < θˆ< H−1(1 − α/2) + θ]
= Pr[F−1(α/2) < θˆ< F−1(1 − α/2)]
= Pr[θˆ< F−1(1 − α/2)] − Pr[θˆ< F−1(α/2)]
= F[F−1(1 − α/2)] − F[F−1(α/2)]
√^ = (1 − α/2) − (α/2) = 1 − α.
- Normal approximation interval uses H−1(q) = zqsvarF(θˆ), (normal).
- α/2 .θˆ− Hˆ−1(1 − α/2), θˆ− Hˆ−1(α/2)Σ≈ We can approximate H(·) via bootstrap: Hˆ(x) = Fˆ(x+ θˆ).
1−α/2.θˆ− Fˆ−1(1 − α/2) + θˆ, θˆ− Fˆ−1(α/2) + θˆΣ= .θˆ− (θˆs
− θˆ), θˆ− (θˆs
− θˆ)Σ.
The post Lecture 4: Bootstrap appeared first on My Assignment Online.