Limited Offer Get 25% off — use code BESTW25
No AI No Plagiarism On-Time Delivery Free Revisions
Claim Now
Prices start at $5 per page · plagiarism-free papers · free revisions for 14 days 24/7 support: support@bestwriters.com
Uncategorized

Lecture 4: Bootstrap

  • Posted
  • Authoradmin
  • Length3,592 words
  • Reading time16 min
Write My Assignment FREE

Lecture 4: Bootstrap

Bootstrap

Bias Correction with Bootstrap Bootstrap Confidence Intervals Bootstrap for Regression

  • Sampling distribution of θˆis hard to derive.

)Or, we just don’t feel like taking the time to.

  • Our sample size is not tiny but is too small for “asymptopia”.

Nonparametric bootstrap: Let Y= [Y, . . . , Y] iidF

  1. ) (some unknown CDF).

1 n ∼Y

    • Fˆ(y)
    n i=1 Let Fˆn(y) = P^r(Y≤ y) = 1ΣnI(Yi≤ y) (fraction of the sample ≤ y)

⇒ = The distribution of the sample approximates the distribution of the population as nincreases.

⇒ = Sampling from a sufficiently large sample approximates sampling from the population.

Parametricbootstrap:► Ifanestimatorθˆnisconsistentandthemodeliscorrect,

p(·|θˆn)approximatesp(·|θ).

=⇒Samplingfromp(·|θˆn)approximatessamplingfromthepopulation.

Notation: Y s = [Y s , . . . , Y s ] the bth “resample” with replacement from observed

b b1 bn ˆ

ˆ vector Y or p(·|θn)

≡ θ(FY) θ the true parameter value or some population quantity function of

FY

Bootstrap

Bias Correction with Bootstrap Bootstrap Confidence Intervals Bootstrap for Regression

Let Y1, . . . , Y

iid

fn ∼Y

(y; θ) = θe−θy, for y> 0 and θ > 0.

Σn

l(θ) = (log θ − θYi) = nlog θ − θnY¯

i =1

Σ► But,

n i=1

Yi∼ Ga(a= n, b= θ)

θˆ= (Y¯)−1

  • Y¯∼ Ga(a= n, b= nθ)
  • n 1 E(θˆ) = E(1/Y¯) = EInvGa(a= n, b= nθ) = nθ > θ

Also, by Jensen’s Inequality, −

  • E(Y¯) = 1/θ (a convex function)

∴ E(θˆ) = E(1/Y¯) > 1/ E(Y¯) = θ.

Recall,

FY population from which Y are drawn

θˆ(FY) population quantity

θˆn(Y) some estimator of θˆ(FY)

biasFY[θˆn(Y)] = EFY[θˆn(Y)] − θˆ(FY)

  • An estimator θˆn(Y) is biasedif biasFY[θˆn(Y)] ƒ= 0
  • Note that Wakefield (2013, Sec. 2.7.1) implicitly defines it the other way, as

θˆ(FY) − EFY[θˆn(Y)].

  • Jensen’s inequality: If it happens that E(Y) = g(θ) for some g, and θˆ= g−1(Y¯), and g(y) is concave or convex, θˆwill be biased for θ.

0.5

n <- 20 # Samplesize

theta.true <- 1 #Exponentialdistributiontruerateparameter

S <- 1000 # Number of replications for the simulation

B <- 999 #Bootstrapresamplesize

#Afunctiontogenerateadataset

mk.data <- function(n, theta=theta.true) rexp(n,theta)

# MLE

theta.mle <- function(y) 1/mean(y)

# MLE sampling distribution

theta.mle.dist <- replicate(S, y.obs <- mk.data(n) theta.mle(y.obs)

)

mean(theta.mle.dist) # Mean of the sampling distribution

## [1] 1.047768

mean(theta.mle.dist)-theta.true #Bias

## [1] 0.04776839

1.0 1.5 2.0 * = mean

b

b

  • Suppose we have a biased estimator:

biasFY[θˆn(Y)] = E[θˆn(Y)] − θˆ(FY) =ƒ 0.

) We don’t know either expectation.

  • Idea:treatFˆnasF.
  • Ys∼Fˆn,soθˆ(Ys)willbebiasedrelativetoθˆ(Y)≡θˆ(Fˆn)byapproximatelythe

b b

same amount:

biasˆn[θˆn(Ys)] = Eˆn[θˆn(Ys)] − θˆn(Y) ≈ EF[θˆn(Y)] − θˆ(FY) = biasF[θˆn(Y)]

F b F b Y

B

B

b

b

cancel some of the bias:

B

b=1

b

b=1
≈1Σθˆn(Ys)−θˆn(Y)

=⇒Subtractingoffθˆn(Ys)−θˆn(Y)=1ΣBθˆn(Ys)−θˆn(Y)fromθˆn(Y)can

θˆn,u(Y)=θˆn(Y)−.θˆn(Ys)−θˆn(Y)Σ =2θˆn(Y)−θˆn(Ys)

# Nonparamametric bootstrap sampling

myboot.np <- function(B, y.obs, theta.f)

replicate(B,

y.b <- sample(y.obs,replace=TRUE) theta.f(y.b)

)

* = mean

# Sampling distribution of the mean of the bootstrap samples

2.0 theta.boot.dist <- replicate(S, y.obs <- mk.data(n)

mean(myboot.np(B, y.obs, theta.mle))

)

1.5mean(theta.boot.dist)-theta.true #Biasedbyabouttwiceasmuch

## [1] 0.1177986

# Debiased estimate and simulation

1.0 theta.bc <- function(theta.obs, theta.boot) theta.obs – (mean(theta.boot) – theta.obs)

0.5

theta.bc.dist <- replicate(S, y.obs <- mk.data(n)

theta.obs <- theta.mle(y.obs)

theta.boot <- myboot.np(B, y.obs, theta.mle)

theta.bc(theta.obs,theta.boot)

)

mean(theta.bc.dist)-theta.true #Muchlessbiased.

## [1] -0.001975362

MLE

Bootstrap

Bias Correction with Bootstrap Bootstrap Confidence Intervals Bootstrap for Regression

Bootstrap

Bias Correction with Bootstrap Bootstrap Confidence Intervals

Basic approaches

Refined approaches Demonstration

Bootstrap for Regression

Wakefield (2013, Sec. 2.7.1); Carpenter and Bithell (2000)

Given

Ys, . . . , Ys

B resamples of the dataset

1 B

b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest

b θˆqsqth quantile of θˆ(Ys)s,

α = 1 − CL

.θ − (θ ˆ ˆs

1−α/2

− θˆ), θˆ− (θˆs

  • θˆ)Σ

α/2

− − − Assumes that distribution of θˆsθˆis the same as the distribution of θˆθ.

0.975 Intuition: 97.5% of the time, θˆwill overestimate θ by less than θˆs

− θˆ,

0.975 so θ will be less than θˆ− (θˆs− θˆ) only 2.5% of the time.

− Often has poor coverage.

Y s, . . . , Y s B resamples of the dataset

1 B

b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest

b θˆqsqth quantile of θˆ(Ys)s,

α = 1 − CL

θˆ(Y) − .θˆ(Ys) − θˆ(Y)Σ ± zs

b

B

b=1

b

√v^arF(θˆ),

b

where θˆ(Ys) = 1ΣBθˆ(Ys) and

1−α/2

^

F θ

B

b=1

θ

b

− B

b=1 θ

b )

. var (ˆ) = 1ΣB .ˆ(Ys) 1ΣB ˆ(Y sΣ2

− Assumes sampling distribution of θˆis symmetric and not too weird.

Intuition: Use bootstrap to debias and estimate the variance.

+ Has good coverage if it is.

^ + Can be useful if var(θˆ) is hard to get analytically.

Y s, . . . , Y s B resamples of the dataset

1 B

b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest

b θˆqsqth quantile of θˆ(Ys)s,

α = 1 − CL

+ Invariant to transformation.

.θˆsα/2, θˆsΣ

1−α/2 − Effectively assumes that there exists a monotone transformation g (θ) s.t.

g(θˆ) ∼ N(g(θ), σ2)

g(θˆs) ∼ N(g(θˆ), σ2)

Intuition: If it exists we could get a 1 − α CI by taking

g−1g(θˆ) ± zsσ, but g(θˆ) + zsσ ≡ g(θˆs

), so we

1−α/2 ˆs

1−α/2

1−α/2

can just use quantiles of θ directly.

)Performs poorly if θˆdoesn’t behave that way (e.g., mean–variance relationship).

Bootstrap

Bias Correction with Bootstrap Bootstrap Confidence Intervals

Basic approaches

Refined approaches Demonstration

Bootstrap for Regression

Y s, . . . , Y s B resamples of the dataset

1 B

b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest

b θˆqsqth quantile of θˆ(Ys)s,

α = 1 − CL

  • A refinement of pivotal interval
  • √ Adjusts for difference between var(θˆs− θˆ) and var(θˆ− θ).
  1. Estimate σˆ ≈ varθ(θˆ).
  2. For each θˆs, estimate σˆs≈

.var ˆs (θˆ)

b

b b θb)E.g., run bootstrap for every θˆs.

)Very computationally intensive, unless a formula exists for σˆ.

  1. Calculate ts(θˆs) = (θˆs− θˆ)/σˆsfor b= 1..B, tsits qth quantile.
  1. 1−α/2 α/2 The CI is then .θˆ+ σˆts, θˆ− σˆtsΣ.

Y s, . . . , Y s B resamples of the dataset

1 B

b θˆ(Ys), b= 1..Bunivariate parameter estimates of interest

b θˆqsqth quantile of θˆ(Ys)s,

α = 1 − CL

  • A refinement of percentile interval

+ Adjusts for skewness in θˆand in how it changes with θ.

) Loosely, assumes that a transformation g (θ) exists s.t.

g(θˆ) ∼ N[g(θ) − b1 + ag(θ), 1 + ag(θ)2]

g(θˆs) ∼ N[g(θˆ) − b1 + ag(θˆ), 1 + ag(θˆ)2]

for a and b.

+ Works very well

− Can misbehave badly if α very small.

  1. Then,qu

= Φ ,b−

s α/2

z 1+a(zs

−b

−b)

,, ql = Φ ,b−

zs

1−α/2 1+a(zs

−b

−b)

,, and the CI is

ˆ)More complicated afor parametric.

l

u.θˆqs, θˆqsΣ.

α/2

1−α/2

Bootstrap

Bias Correction with Bootstrap Bootstrap Confidence Intervals

Basic approaches

Refined approaches Demonstration

Bootstrap for Regression

# Asymptotic CI using I(theta):

theta.asymp.CI <- function(n, theta.obs) theta.obs+qnorm(c(.025,.975))*sqrt(theta.obs^2/n)


# Bootstrap CI using normal approximation:

theta.boot.N.CI <- function(n, theta.obs, theta.boot) bias <- mean(theta.boot) – theta.obs

theta.obs – bias + qnorm(c(.025,.975))*sd(theta.boot)


# Bootstrap CI using quantiles:

theta.boot.Q.CI <- function(n, theta.obs, theta.boot)

quantile(theta.boot,c(0.025,0.975))

# Bootstrap CI using BCa:

theta.boot.BCa.CI <- function(n, theta.obs, theta.boot) b <- qnorm(mean(theta.boot<theta.obs))

a <- – mean((theta.boot-mean(theta.boot))^3)/sd(theta.boot)^3 / 6 z.minus.b <- qnorm(c(.975,.025))-b

q <- pnorm(b-z.minus.b/(1+a*z.minus.b)) quantile(theta.boot, q)

rbind(Asymptotic=theta.asymp.CI(n, theta.obs), Normal=theta.boot.N.CI(n, theta.obs, theta.boot), Quantile=theta.boot.Q.CI(n, theta.obs, theta.boot), Basic=theta.boot.pivot.CI(n, theta.obs, theta.boot), BCa=theta.boot.BCa.CI(n, theta.obs, theta.boot))

## 2.5 97.5

## Asymptotic 0.5690201 1.456904

## Normal 0.5019225 1.430258

## Quantile 0.6873244 1.605447

## Basic 0.4204773 1.338600

## BCa 0.5725606 1.466644

  • Automates bootstrapping and CI calculation
  • For ordinary (default) bootstrap, “statistic” is a function of the original dataset and indices selecting the resample.

library(boot)

theta.Rboot <- boot(y.obs, #data

function(y.obs,i) theta.mle(y.obs[i]), B) #Statistic

boot.ci(theta.Rboot) # Get theCIs

## Warning in boot.ci(theta.Rboot): bootstrap variances needed for studentized intervals

## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS

## Based on 999 bootstrap replicates ##

## CALL :

## boot.ci(boot.out = theta.Rboot) ##

## Intervals :

## Level Normal Basic

## 95 ( 0.447, 1.469 ) ( 0.357, 1.348 ) ##

## Level Percentile BCa

## 95 ( 0.678, 1.669 ) ( 0.644, 1.559 )

## Calculations and Intervals on Original Scale

  • A coverage simulation examines whether a statistical method produces good confidence intervals under a variety of assumptions.
  1. Simulate Y from some known parameter θ.
  2. Calculate a CI for θ based on Y.
  3. How often does the CI actually contain θ? A 95% CI should contain it 95% of the time, etc..

S <- 1000 # Simulationsize

covers <- function(CI, truth) CI[1]<=truth && CI[2]>=truth

cover.sim <- function(mk.data.f, boot.f)

replicate(S,

y.obs <- mk.data.f(n)

theta.boot <- boot.f(B, y.obs, theta.mle) theta.obs <- theta.mle(y.obs)

c(Asymptotic=covers(theta.asymp.CI(n, theta.obs),theta.true),

`Normal`=covers(theta.boot.N.CI(n, theta.obs, theta.boot),theta.true),

`Quantile`=covers(theta.boot.Q.CI(n, theta.obs, theta.boot),theta.true),

`Basic`=covers(theta.boot.pivot.CI(n, theta.obs, theta.boot),theta.true),

`BCa`=covers(theta.boot.BCa.CI(n, theta.obs, theta.boot),theta.true))

)

rowMeans(cover.sim(mk.data, myboot.np))
## Asymptotic Normal Quantile Basic BCa
## 0.953 0.939 0.893 0.891 0.910

theta.boot <- myboot.p(B, y.obs, theta.mle)

#ThisBCaOKinthiscase,butiffyingeneral rbind(Asymptotic=theta.asymp.CI(n, theta.obs), Normal=theta.boot.N.CI(n, theta.obs, theta.boot), Quantile=theta.boot.Q.CI(n, theta.obs, theta.boot), Basic=theta.boot.pivot.CI(n, theta.obs, theta.boot), BCa=theta.boot.BCa.CI(n, theta.obs, theta.boot))

## 2.5 97.5

## Asymptotic 0.5690201 1.456904

## Normal 0.4853627 1.449853

## Quantile 0.6847231 1.647648

## Basic 0.3782760 1.341201

## BCa 0.5562614 1.385008

rowMeans(cover.sim(mk.data, myboot.p))
## Asymptotic Normal Quantile Basic BCa
## 0.952 0.963 0.936 0.918 0.942
  • statistic is just a function of data.
  • ran.gen is a function of original data and parameters passed via mle that generates a replication of the dataset.
  • BCa is not as straightforward, because a’s estimation depends on model properties. (This workaround might be a bit iffy.)

theta.Rboot <- boot(y.obs, theta.mle, B, sim=”parametric”, ran.gen=function(d, p) mk.data(length(d), p),mle=theta.obs)

boot.ci(theta.Rboot, L=empinf(data=y.obs, stype=”i”, type=”jack”, statistic=function(y.obs,

## Warning in boot.ci(theta.Rboot, L = empinf(data = y.obs, stype = “i”, type = “jack”, : bootstrap variances needed for studentized intervals

## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS

## Based on 999 bootstrap replicates ##

## CALL :

## boot.ci(boot.out = theta.Rboot, L = empinf(data = y.obs, stype = “i”, ## type = “jack”, statistic = function(y.obs, i) theta.mle(y.obs[i]))) ##

## Intervals :

## Level Normal Basic

## 95 ( 0.449, 1.469 ) ( 0.298, 1.326 ) ##

## Level Percentile BCa

## 95 ( 0.700, 1.728 ) ( 0.645, 1.523 )

## Calculations and Intervals on Original Scale ## Some BCa intervals may be unstable

  • Now, suppose that we were wrong about our data: our distribution was, in fact, Ga(a= 4, b= 4), not exponential: same mean, but smaller SD.

mk.data2 <- function(n) rgamma(n,4,theta.true*4)

Nonparametric bootstrap:

rowMeans(cover.sim(mk.data2, myboot.np))
## Asymptotic Normal Quantile Basic BCa
## 1.000 0.932 0.915 0.917 0.920
Parametric bootstrap:
rowMeans(cover.sim(mk.data2, myboot.p))
## Asymptotic Normal Quantile Basic BCa
## 1.000 1.000 1.000 0.993 0.999

Bootstrap

Bias Correction with Bootstrap Bootstrap Confidence Intervals Bootstrap for Regression

Wakefield (2013, Sec. 2.7.2)

We know what to do with Yi s, but what about xi s?

  • Resample cases ((Yi, xi))

+ Generally robust to, e.g. variance assumptions.

− Ignores the sampling design.

)If the sampling is blocked, stratified, etc., bootstrap resampling can be modified to reflect that.

  • | − | Resample responses Yi xi (parametric) or residuals (Yi µi(β)) xi, β

) Residuals can also be resampled parametrically.

) If the model postulates a mean-variance relationship, one can resample studentised (divided by predicted variance) residuals.

+ Preserves sampling design.

+ Robust to distributional assumptions (if nonparametric).

− Not robust to mean-variance relationship assumptions.

  • Nonparametric resampling of residuals would be problematic, since the variance increases with the mean.
  • Parametric bootstrap produces results comparable to the likelihood standard errors, while resampling cases produces results comparable to the sandwich standard errors.

load(“Radon_Example.RData”)

#Afunctiontofitthemodelofinterest,usingdata,selectingrowsspecifiedbyinds(th

do.poisfit <- function(data, inds=seq_len(nrow(data)), coef=TRUE) data <- data[inds,] # Select only the specified rows

poisfit <- glm(Obs~offset(log(Exp))+radon,family=poisson, na.action=na.omit,data=data) ##Ifcoef=TRUE,extractthecoefficients.Otherwise,returnthefullfitobject. if(coef) coef(poisfit) elsepoisfit

# The MLE

poisfit <- do.poisfit(MNrad,coef=FALSE) theta.obs <- coef(poisfit)

n <- nrow(MNrad)

ms <- predict(poisfit, type=”response”)

MNrad.boot <- MNrad

theta.boot <- t(replicate(B,

MNrad.boot$Obs <- rpois(length(ms), ms) # Generate new cancercounts

do.poisfit(MNrad.boot)

))

cov(theta.boot)

## (Intercept) radon ## (Intercept) 0.0007113803 -1.349959e-04 ## radon -0.0001349959 2.855784e-05

vcov(poisfit)

## (Intercept) radon ## (Intercept) 0.0007186984 -1.370103e-04 ## radon -0.0001370103 2.913701e-05

lapply(1:2, function(i) rbind(Asymptotic=coef(poisfit)[i]+qnorm(c(.025,.975))*sqrt(diag(vco

Normal=theta.boot.N.CI(n, theta.obs[i], theta.boot[,i]), Quantile=theta.boot.Q.CI(n, theta.obs[i], theta.boot[,i]), Basic=theta.boot.pivot.CI(n, theta.obs[i], theta.boot[,i]), BCa=theta.boot.BCa.CI(n, theta.obs[i], theta.boot[,i])))

## [[1]]

## 2.5 97.5

## Asymptotic 0.1180851 0.2231727

## Normal 0.1184123 0.2229635

## Quantile 0.1201091 0.2229689

## Basic 0.1182889 0.2211487

## BCa 0.1219132 0.2237508

##

## [[2]]

## 2.5 97.5

## Asymptotic -0.04673742 -0.02557815

## Normal -0.04658703 -0.02563912

## Quantile -0.04676029 -0.02589079

## Basic -0.04642478 -0.02555528

## BCa -0.04681753 -0.02627902

  • You have to “ask” for each variable at a time.

theta.boot <- boot(MNrad, statistic=do.poisfit, sim=”parametric”, R=B, ran.gen = function( boot.ci(theta.boot, index=2, L=empinf(data=MNrad, stype=”i”, type=”jack”, statistic=do.po

## Warning in boot.ci(theta.boot, index = 2, L = empinf(data = MNrad, stype = “i”,

: bootstrap variances needed for studentized intervals

## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS

## Based on 999 bootstrap replicates ##

## CALL :

## boot.ci(boot.out = theta.boot, index = 2, L = empinf(data = MNrad, ## stype = “i”, type = “jack”, statistic = do.poisfit))

##

## Intervals :

## Level Normal Basic

## 95 (-0.0465, -0.0258 ) (-0.0467, -0.0259 ) ##

## Level Percentile BCa

## 95 (-0.0464, -0.0257 ) (-0.0458, -0.0245 )

## Calculations and Intervals on Original Scale

theta.boot <- t(replicate(B,

#Wewantwhole(Obs,Exp,x)triplets,soweneedtoselectrowsinthedataframe.

inds <- sample.int(nrow(MNrad),replace=TRUE) do.poisfit(MNrad,inds)

))

cov(theta.boot)

## (Intercept) radon ## (Intercept) 0.0021112536 -3.423497e-04 ## radon -0.0003423497 7.314646e-05

vcov(poisfit)

## (Intercept) radon ## (Intercept) 0.0007186984 -1.370103e-04 ## radon -0.0001370103 2.913701e-05

sandwich(poisfit)

## (Intercept) radon ## (Intercept) 0.0018419895 -2.985599e-04 ## radon -0.0002985599 6.438377e-05

lapply(1:2, function(i) rbind(Asymptotic=coef(poisfit)[i]+qnorm(c(.025,.975))*sqrt(diag(vco

Normal=theta.boot.N.CI(n, theta.obs[i], theta.boot[,i]), Quantile=theta.boot.Q.CI(n, theta.obs[i], theta.boot[,i]), Basic=theta.boot.pivot.CI(n, theta.obs[i], theta.boot[,i]), BCa=theta.boot.BCa.CI(n, theta.obs[i], theta.boot[,i])))

## [[1]]

## 2.5 97.5

## Asymptotic 0.11808508 0.2231727

## Normal 0.08484135 0.2649557

## Quantile 0.07699282 0.2497400

## Basic 0.09151778 0.2642649

## BCa 0.08353562 0.2594277

##

## [[2]]

## 2.5 97.5

## Asymptotic -0.04673742 -0.02557815

## Normal -0.05277069 -0.01924523

## Quantile -0.05347350 -0.02025274

## Basic -0.05206283 -0.01884207

## BCa -0.05344392 -0.02020988

theta.boot <- boot(MNrad, statistic=do.poisfit, sim=”ordinary”, R=B)

suppressWarnings(boot.ci(theta.boot, index=2))

## BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS

## Based on 999 bootstrap replicates ##

## CALL :

## boot.ci(boot.out = theta.boot, index = 2) ##

## Intervals :

## Level Normal Basic

## 95 (-0.0527, -0.0200 ) (-0.0535, -0.0200 ) ##

## Level Percentile BCa

## 95 (-0.0523, -0.0188 ) (-0.0523, -0.0184 )

## Calculations and Intervals on Original Scale

  • Very robust and flexible, but tends to have slightly low coverage, particularly nonparametric.
  • Must reflect the sampling process, e.g.,

Stratification: Resample from each stratum in accordance with its sample size. Serial (temporal) dependence: Use block bootstrap methods.

James Carpenter and John Bithell. Bootstrap confidence intervals: When, which, what? A practical guide for medical statisticians. Statistics in Medicine, 19: 1141–1164, 2000.

Anthony C. Davison and David V. Hinkley. Bootstrap Methods and their Application. Cambridge University Press, Cambridge, 1997. ISBN 0-521-57391-2. URL http://statwww.epfl.ch/davison/BMA/.

Jon Wakefield. BayesianandFrequentistRegressionMethods. Springer Series in Statistics. Springer New York, 2013. ISBN 978-1-4419-0925-1. doi:10.1007/978-1-4419-0925-1.

Let Xbe a r.v. that is not degenerate(can have multiple values), and let ϕ(x) be a smooth function of xin the support of X(i.e., possible values of X).

  • ϕE(X) = Eϕ(X) if ϕ(x) is linear (i.e., ϕjj(x) = 0 in the support of X)
  • X) ϕE(X) < Eϕ(X) if ϕ(x) is strictly convex (i.e., ϕjj(x) > 0 in the support of
  • X) ϕE(X) > Eϕ(X) if ϕ(x) is strictly concave (i.e., ϕjj(x) < 0 in the support of

Similarly with sample means: if ϕ(x) ≡ 1Σn

ϕ(xi ), if not all xi s are the same:

n i=1

  • ϕx¯ = ϕ(x) if ϕ(x) is linear
  • ϕx¯ < ϕ(x) if ϕ(x) is strictly convex
  • ϕx¯ > ϕ(x) if ϕ(x) is strictly concave

i=1

Qn Example: For xi> 0 not all the same, geometric mean (

xi)1/nis smaller than

n

i=1

1. Let ϕ(x) = log(x). ϕjj(x) = −x−2< 0.
the arithmetic mean 1 Σn xi :

  1. (Qnxi)1/n= exp[log(Qn

n

i=1

xi)1/n] = exp[ 1Σnlogxi].

  1. i=1 i=1 i=1 i=1 1 Σn logxi < log 1Σn

xi = logx¯ (Jensen’s Inequality)

  1. Σn exp(x) is strictly increasing, so

n

i=1x¯ = explog(x¯) > exp[ 1

logxi] = (Qn

xi)1/n.

Wakefield (2013, Ex. 2.9)

  • − − − Consider an interval in the form of (θˆH−1(1 α/2), θˆH−1(α/2)), where
  • −

H( ) is the CDF of the sampling distribution of θˆθ.

    • −
    If F( ) is the CDF of θˆ, then H(x) = F(x+ θ), so H−1(q) = F−1(q) θ and

H−1(q) + θ = F−1(q).

  • Then,

Pr [θˆ− H−1(1 − α/2) < θ < θˆ− H−1(α/2)]

= Pr[−H−1(1 − α/2) − θ < −θˆ< −H−1(α/2) − θ]

= Pr[H−1(α/2) + θ < θˆ< H−1(1 − α/2) + θ]

= Pr[F−1(α/2) < θˆ< F−1(1 − α/2)]

= Pr[θˆ< F−1(1 − α/2)] − Pr[θˆ< F−1(α/2)]

= F[F−1(1 − α/2)] − F[F−1(α/2)]

√^ = (1 − α/2) − (α/2) = 1 − α.

  • Normal approximation interval uses H−1(q) = zqsvarF(θˆ), (normal).
  • α/2 .θˆ− Hˆ−1(1 − α/2), θˆ− Hˆ−1(α/2)Σ≈ We can approximate H(·) via bootstrap: Hˆ(x) = Fˆ(x+ θˆ).

1−α/2.θˆ− Fˆ−1(1 − α/2) + θˆ, θˆ− Fˆ−1(α/2) + θˆΣ= .θˆ− (θˆs

− θˆ), θˆ− (θˆs

− θˆ)Σ.

The post Lecture 4: Bootstrap appeared first on My Assignment Online.

Plagiarism Free Assignment Help

Expert Help With This Assignment — On Your Terms

✓ Native UK, USA & Australia writers ✓ Deadline from 3 hours ✓ 100% Plagiarism-Free — Turnitin included ✓ Unlimited free revisions ✓ Free to submit — compare quotes

admin

Academic writer and editor at BestWriters.

Get your paper off your plate tonight

Post your instructions for free. Prices start at $5 per page.

Place an order