Download Bayesian and Frequentist Regression Methods by Jon Wakefield PDF

By Jon Wakefield

This publication offers a balanced, smooth precis of Bayesian and frequentist equipment for regression analysis.

Table of Contents


Bayesian and Frequentist Regression Methods

ISBN 9781441909244 ISBN 9781441909251



Chapter 1 advent and Motivating Examples

1.1 Introduction
1.2 version Formulation
1.3 Motivating Examples
1.3.1 Prostate Cancer
1.3.2 consequence After Head Injury
1.3.3 Lung melanoma and Radon
1.3.4 Pharmacokinetic Data
1.3.5 Dental Growth
1.3.6 Spinal Bone Mineral Density
1.4 Nature of Randomness
1.5 Bayesian and Frequentist Inference
1.6 the administrative Summary
1.7 Bibliographic Notes

Part I

bankruptcy 2 Frequentist Inference
2.1 Introduction
2.2 Frequentist Criteria
2.3 Estimating Functions
2.4 Likelihood
o 2.4.1 greatest chance Estimation
o 2.4.2 versions on Likelihood
o 2.4.3 version Misspecification
2.5 Quasi-likelihood 2.5.1 greatest Quasi-likelihood Estimation
o 2.5.2 A extra advanced Mean-Variance Model
2.6 Sandwich Estimation
2.7 Bootstrap Methods
o 2.7.1 The Bootstrap for a Univariate Parameter
o 2.7.2 The Bootstrap for Regression
o 2.7.3 Sandwich Estimation and the Bootstrap
2.8 collection of Estimating Function
2.9 speculation Testing
o 2.9.1 Motivation
o 2.9.2 Preliminaries
o 2.9.3 rating Tests
o 2.9.4 Wald Tests
o 2.9.5 chance Ratio Tests
o 2.9.6 Quasi-likelihood
o 2.9.7 comparability of try out Statistics
2.10 Concluding Remarks
2.11 Bibliographic Notes
2.12 Exercises
bankruptcy three Bayesian Inference
3.1 Introduction
3.2 The Posterior Distribution and Its Summarization
3.3 Asymptotic houses of Bayesian Estimators
3.4 past Choice
o 3.4.1 Baseline Priors
o 3.4.2 sizeable Priors
o 3.4.3 Priors on significant Scales
o 3.4.4 Frequentist Considerations
3.5 version Misspecification
3.6 Bayesian version Averaging
3.7 Implementation
o 3.7.1 Conjugacy
o 3.7.2 Laplace Approximation
o 3.7.3 Quadrature
o 3.7.4 built-in Nested Laplace Approximations
o 3.7.5 value Sampling Monte Carlo
o 3.7.6 Direct Sampling utilizing Conjugacy
o 3.7.7 Direct Sampling utilizing the Rejection Algorithm
3.8 Markov Chain Monte Carlo 3.8.1 Markov Chains for Exploring Posterior Distributions
o 3.8.2 The Metropolis-Hastings Algorithm
o 3.8.3 The city Algorithm
o 3.8.4 The Gibbs Sampler
o 3.8.5 Combining Markov Kernels: Hybrid Schemes
o 3.8.6 Implementation Details
o 3.8.7 Implementation Summary
3.9 Exchangeability
3.10 speculation checking out with Bayes Factors
3.11 Bayesian Inference in line with a Sampling Distribution
3.12 Concluding Remarks
3.13 Bibliographic Notes
3.14 Exercises
bankruptcy four speculation trying out and Variable Selection
4.1 Introduction
4.2 Frequentist speculation Testing
o 4.2.1 Fisherian Approach
o 4.2.2 Neyman-Pearson Approach
o 4.2.3 Critique of the Fisherian Approach
o 4.2.4 Critique of the Neyman-Pearson Approach
4.3 Bayesian speculation trying out with Bayes components 4.3.1 evaluation of Approaches
o 4.3.2 Critique of the Bayes issue Approach
o 4.3.3 A Bayesian View of Frequentist speculation Testing
4.4 The Jeffreys-Lindley Paradox
4.5 trying out a number of Hypotheses: normal Considerations
4.6 trying out a number of Hypotheses: fastened variety of Tests
o 4.6.1 Frequentist Analysis
o 4.6.2 Bayesian Analysis
4.7 checking out a number of Hypotheses: Variable Selection
4.8 ways to Variable choice and Modeling
o 4.8.1 Stepwise Methods
o 4.8.2 All attainable Subsets
o 4.8.3 Bayesian version Averaging
o 4.8.4 Shrinkage Methods
4.9 version construction Uncertainty
4.10 a realistic Compromise to Variable Selection
4.11 Concluding Comments
4.12 Bibliographic Notes
4.13 Exercises

Part II

bankruptcy five Linear Models
5.1 Introduction
5.2 Motivating instance: Prostate Cancer
5.3 version Specifiation
5.4 A Justificatio for Linear Modeling
5.5 Parameter Interpretation
o 5.5.1 Causation as opposed to Association
o 5.5.2 a number of Parameters
o 5.5.3 facts Transformations
5.6 Frequentist Inference 5.6.1 Likelihood
o 5.6.2 Least Squares Estimation
o 5.6.3 The Gauss-Markov Theorem
o 5.6.4 Sandwich Estimation
5.7 Bayesian Inference
5.8 research of Variance
o 5.8.1 One-Way ANOVA
o 5.8.2 Crossed Designs
o 5.8.3 Nested Designs
o 5.8.4 Random and combined results Models
5.9 Bias-Variance Trade-Off
5.10 Robustness to Assumptions
o 5.10.1 Distribution of Errors
o 5.10.2 Nonconstant Variance
o 5.10.3 Correlated Errors
5.11 evaluation of Assumptions
o 5.11.1 overview of Assumptions
o 5.11.2 Residuals and In uence
o 5.11.3 utilizing the Residuals
5.12 instance: Prostate Cancer
5.13 Concluding Remarks
5.14 Bibliographic Notes
5.15 Exercises
bankruptcy 6 basic Regression Models
6.1 Introduction
6.2 Motivating instance: Pharmacokinetics of Theophylline
6.3 Generalized Linear Models
6.4 Parameter Interpretation
6.5 chance Inference for GLMs 6.5.1 Estimation
o 6.5.2 Computation
o 6.5.3 speculation Testing
6.6 Quasi-likelihood Inference for GLMs
6.7 Sandwich Estimation for GLMs
6.8 Bayesian Inference for GLMs
o 6.8.1 earlier Specification
o 6.8.2 Computation
o 6.8.3 speculation Testing
o 6.8.4 Overdispersed GLMs
6.9 overview of Assumptions for GLMs
6.10 Nonlinear Regression Models
6.11 Identifiabilit
6.12 chance Inference for Nonlinear versions 6.12.1 Estimation
o 6.12.2 speculation Testing
6.13 Least Squares Inference
6.14 Sandwich Estimation for Nonlinear Models
6.15 The Geometry of Least Squares
6.16 Bayesian Inference for Nonlinear Models
o 6.16.1 past Specification
o 6.16.2 Computation
o 6.16.3 speculation Testing
6.17 overview of Assumptions for Nonlinear Models
6.18 Concluding Remarks
6.19 Bibliographic Notes
6.20 Exercises
bankruptcy 7 Binary information Models
7.1 Introduction
7.2 Motivating Examples 7.2.1 final result After Head Injury
o 7.2.2 airplane Fasteners
o 7.2.3 Bronchopulmonary Dysplasia
7.3 The Binomial Distribution 7.3.1 Genesis
o 7.3.2 infrequent Events
7.4 Generalized Linear versions for Binary info 7.4.1 Formulation
o 7.4.2 hyperlink Functions
7.5 Overdispersion
7.6 Logistic Regression types 7.6.1 Parameter Interpretation
o 7.6.2 probability Inference for Logistic Regression Models
o 7.6.3 Quasi-likelihood Inference for Logistic Regression Models
o 7.6.4 Bayesian Inference for Logistic Regression Models
7.7 Conditional probability Inference
7.8 review of Assumptions
7.9 Bias, Variance, and Collapsibility
7.10 Case-Control Studies
o 7.10.1 The Epidemiological Context
o 7.10.2 Estimation for a Case-Control Study
o 7.10.3 Estimation for a Matched Case-Control Study
7.11 Concluding Remarks
7.12 Bibliographic Notes
7.13 Exercises

Part III

bankruptcy eight Linear Models
8.1 Introduction
8.2 Motivating instance: Dental development Curves
8.3 The Effciency of Longitudinal Designs
8.4 Linear combined versions 8.4.1 the overall Framework
o 8.4.2 Covariance versions for Clustered Data
o 8.4.3 Parameter Interpretation for Linear combined Models
8.5 probability Inference for Linear combined Models
o 8.5.1 Inference for fastened Effects
o 8.5.2 Inference for Variance elements through greatest Likelihood
o 8.5.3 Inference for Variance elements through limited greatest Likelihood
o 8.5.4 Inference for Random Effects
8.6 Bayesian Inference for Linear combined versions 8.6.1 A Three-Stage Hierarchical Model
o 8.6.2 Hyperpriors
o 8.6.3 Implementation
o 8.6.4 Extensions
8.7 Generalized Estimating Equations 8.7.1 Motivation
o 8.7.2 The GEE Algorithm
o 8.7.3 Estimation of Variance Parameters
8.8 evaluate of Assumptions 8.8.1 overview of Assumptions
o 8.8.2 ways to Assessment
8.9 Cohort and Longitudinal Effects
8.10 Concluding Remarks
8.11 Bibliographic Notes
8.12 Exercises
bankruptcy nine common Regression Models
9.1 Introduction
9.2 Motivating Examples
o 9.2.1 birth control Data
o 9.2.2 Seizure Data
o 9.2.3 Pharmacokinetics of Theophylline
9.3 Generalized Linear combined Models
9.4 probability Inference for Generalized Linear combined Models
9.5 Conditional probability Inference for Generalized Linear combined Models
9.6 Bayesian Inference for Generalized Linear combined types 9.6.1 version Formulation
o 9.6.2 Hyperpriors
9.7 Generalized Linear combined versions with Spatial Dependence 9.7.1 A Markov Random box Prior
o 9.7.2 Hyperpriors
9.8 Conjugate Random results Models
9.9 Generalized Estimating Equations for Generalized Linear Models
9.10 GEE2: attached Estimating Equations
9.11 Interpretation of Marginal and Conditional Regression Coeffiients
9.12 advent to Modeling based Binary Data
9.13 combined versions for Binary facts 9.13.1 Generalized Linear combined versions for Binary Data
o 9.13.2 chance Inference for the Binary combined Model
o 9.13.3 Bayesian Inference for the Binary combined Model
o 9.13.4 Conditional probability Inference for Binary combined Models
9.14 Marginal versions for established Binary Data
o 9.14.1 Generalized Estimating Equations
o 9.14.2 Loglinear Models
o 9.14.3 additional Multivariate Binary Models
9.15 Nonlinear combined Models
9.16 Parameterization of the Nonlinear Model
9.17 chance Inference for the Nonlinear combined Model
9.18 Bayesian Inference for the Nonlinear combined Model
o 9.18.1 Hyperpriors
o 9.18.2 Inference for capabilities of Interest
9.19 Generalized Estimating Equations
9.20 overview of Assumptions for common Regression Models
9.21 Concluding Remarks
9.22 Bibliographic Notes
9.23 Exercises

Part IV

bankruptcy 10 Preliminaries for Nonparametric Regression
10.1 Introduction
10.2 Motivating Examples
o 10.2.1 mild Detection and Ranging
o 10.2.2 Ethanol Data
10.3 The optimum Prediction
o 10.3.1 non-stop Responses
o 10.3.2 Discrete Responses with ok Categories
o 10.3.3 normal Responses
o 10.3.4 In Practice
10.4 Measures of Predictive Accuracy
o 10.4.1 non-stop Responses
o 10.4.2 Discrete Responses with ok Categories
o 10.4.3 basic Responses
10.5 a primary examine Shrinkage Methods
o 10.5.1 Ridge Regression
o 10.5.2 The Lasso
10.6 Smoothing Parameter Selection
o 10.6.1 Mallows CP
o 10.6.2 K-Fold Cross-Validation
o 10.6.3 Generalized Cross-Validation
o 10.6.4 AIC for normal Models
o 10.6.5 Cross-Validation for Generalized Linear Models
10.7 Concluding Comments
10.8 Bibliographic Notes
10.9 Exercises
bankruptcy eleven Spline and Kernel Methods
11.1 Introduction
11.2 Spline equipment 11.2.1 Piecewise Polynomials and Splines
o 11.2.2 average Cubic Splines
o 11.2.3 Cubic Smoothing Splines
o 11.2.4 B-Splines
o 11.2.5 Penalized Regression Splines
o 11.2.6 a quick Spline Summary
o 11.2.7 Inference for Linear Smoothers
o 11.2.8 Linear combined version Spline illustration: probability Inference
o 11.2.9 Linear combined version Spline illustration: Bayesian Inference
11.3 Kernel Methods
o 11.3.1 Kernels
o 11.3.2 Kernel Density Estimation
o 11.3.3 The Nadaraya-Watson Kernel Estimator
o 11.3.4 neighborhood Polynomial Regression
11.4 Variance Estimation
11.5 Spline and Kernel equipment for Generalized Linear Models
o 11.5.1 Generalized Linear types with Penalized Regression Splines
o 11.5.2 A Generalized Linear combined version Spline Representation
o 11.5.3 Generalized Linear versions with neighborhood Polynomials
11.6 Concluding Comments
11.7 Bibliographic Notes
11.8 Exercises
bankruptcy 12 Nonparametric Regression with a number of Predictors
12.1 Introduction
12.2 Generalized Additive versions 12.2.1 version Formulation
o 12.2.2 Computation through Backfittin
12.3 Spline equipment with a number of Predictors
o 12.3.1 common skinny Plate Splines
o 12.3.2 skinny Plate Regression Splines
o 12.3.3 Tensor Product Splines
12.4 Kernel tools with a number of Predictors
12.5 Smoothing Parameter Estimation 12.5.1 traditional Approaches
o 12.5.2 combined version Formulation
12.6 Varying-Coefficien Models
12.7 Regression timber 12.7.1 Hierarchical Partitioning
o 12.7.2 a number of Adaptive Regression Splines
12.8 Classificatio
o 12.8.1 Logistic types with okay Classes
o 12.8.2 Linear and Quadratic Discriminant Analysis
o 12.8.3 Kernel Density Estimation and Classificatio
o 12.8.4 Classificatio Trees
o 12.8.5 Bagging
o 12.8.6 Random Forests
12.9 Concluding Comments
12.10 Bibliographic Notes
12.11 Exercises

Part V

Appendix A Differentiation of Matrix Expressions
Appendix B Matrix Results
Appendix C a few Linear Algebra
Appendix D chance Distributions and producing Functions
Appendix E services of ordinary Random Variables
Appendix F a few effects from Classical Statistics
Appendix G simple huge pattern Theory



Show description

Read or Download Bayesian and Frequentist Regression Methods PDF

Similar probability books

Nonparametric Regression and Spline Smoothing

This textbook for a graduate point introductory direction on information smoothing covers sequence estimators, kernel estimators, smoothing splines, and least-squares splines. the hot version deletes many of the asymptotic idea for smoothing splines and smoothing spline versions, and provides order choice for hierarchical versions, estimation in in part linear types, polynomial-trigonometric regression, new effects on bandwidth choice, and in the community linear regression.

Interest Rate Models: an Infinite Dimensional Stochastic Analysis Perspective (Springer Finance)

Rate of interest versions: an enormous Dimensional Stochastic research standpoint experiences the mathematical concerns that come up in modeling the rate of interest time period constitution. those matters are approached through casting the rate of interest types as stochastic evolution equations in countless dimensional functionality areas.

Linear model theory. Univariate, multivariate, and mixed models

An exact and available presentation of linear version concept, illustrated with facts examples Statisticians frequently use linear types for info research and for constructing new statistical equipment. so much books at the topic have traditionally mentioned univariate, multivariate, and combined linear types individually, while Linear version concept: Univariate, Multivariate, and combined versions offers a unified therapy that allows you to clarify the differences one of the 3 periods of versions.

Additional resources for Bayesian and Frequentist Regression Methods

Example text

13. The above gives one a way of thinking about where the random terms in models arise from, namely as unmeasured covariates. In terms of distributional assumptions, some distributions arise naturally as a consequence of simple physical models. For example, suppose we are interested in modeling the number of events occurring over time. The process we now describe has been found empirically to model a number of phenomena, for example the arrival of calls at a telephone exchange or the emission of particles from a radioactive source.

7, describe methods for determining properties of the estimator that do not depend on correct specification of the full probability model. Example: Binomial Likelihood For a single observation from a binomial distribution, Y | p ∼ Binomial(n, p), the log-likelihood is l(p) = Y log p + (n − Y ) log(1 − p), where we omit the term log n Y because it is constant with respect to p. The score is S(p) = dl Y n−Y = − , dp p 1−p and setting S(p) = 0 gives p = Y /n.

L(θ) = −E ∂θ∂θ T ∂θ T Result. 14) ∂S(θ) = E [S(θ)S(θ)T ] . 15) E[S(θ)] = E and In (θ) = −E Proof. For simplicity we give a prove for the situation in which θ is univariate, and the observations are independent and identically distributed. Under these circumstances In (θ) = nI1 (θ), where I1 (θ) = −E d2 log p(Y | θ) . 14). 15). Viewing the score as an estimating function, Gn (θ) = 1 1 S(θ) = n n n i=1 d log p(Yi | θ), dθ shows that the MLE satisfies Gn (θn ) = 0. 1 of Sect. 3, we require A(θ) = E ∂ ∂2 G(θ, Y ) = E log p(Y | θ) T ∂θ ∂θ∂θ T and B(θ) = E [G(θ, Y )G(θ, Y )T ] = E ∂ log p(Y | θ) ∂θ ∂ log p(Y | θ) ∂θ T .

Download PDF sample

Rated 4.54 of 5 – based on 11 votes