Maximum Likelihood Estimator: The Untold Stories and Caveats for Application

Background and Purpose

Social worker researchers are increasingly applying advanced statistical models in their quantitative research. Regardless of which model they use (structural equation modeling, hierarchical linear modeling, generalized estimating equation, logistic regression, Poisson regression, negative binomial regression, parametric survival models, Cox proportional hazards model, Heckman sample selection model, propensity score analysis, etc.), the contemporary statistical analysis entirely relies on maximum likelihood estimator (MLE) to estimate unknown parameters. Due to the complexity and highly technical nature of the numerical approach embedded in MLE, textbooks typically describe MLE in an oversimplified fashion. The untold stories about MLE create barriers, anxieties, and uncertainties for users to conduct statistical analysis, cause the users to fail in obtaining results (i.e., to experience the problem of “nonconvergence of model”), and even run the risk for users to interpret results incorrectly.  This workshop aims to review the mathematical principles and basic setups of MLE by using an intuitive approach (Excel) so that users can see what happens inside the black box. Implications, particularly caveats for running MLE, are summarized.

Contents

The workshop focuses on the following topics: (1) the objective of optimization (i.e., minimization or maximization) in statistical modeling – minimizing errors through ordinary-least- squares and maximizing the likelihood of reproducing sample through MLE; (2) likelihood function and five basic steps in MLE; (3) overview of numerical approaches – the workshop focuses on  the Newton-Raphson algorithm to describe the basic ideas of MLE, though it also highlights key features of related algorithms (e.g., the BHHH algorithm); (4) overview of the MLE procedures offered by popular statistical software packages (SPSS, Stata, SAS, Mplus); and (5) application issues and caveats.

Pedagogical Techniques

The workshop takes two examples (i.e., the logistic regression and Cox regression) to illustrate the core idea and basic steps in MLE. It employs a carefully designed spreadsheet program (Excel) to demonstrate the estimation of unknown parameters and how the process helps users accomplish the goal of maximizing a likelihood function. The spreadsheet program also helps participants understand complex concepts of MLE, such as gradient/score vector, the Hessian matrix, calculation of direction vector, and calculation of stepsize.

Implications

Through a nontechnical and intuitive explanation of MLE, participants will learn from this workshop: (1) when and why a model fails to converge or experiences nonconcavity; (2) tips for troubleshooting – understand the utilities of number of iterations, convergence criterion, initial values, and caveats when employing these functions; (3) basic requirements for running MLE – minimum sample size, the importance of having a substantively meaningful while statistically parsimonious model, caveats for conducting likelihood ratio test; and (4) common pitfalls in statistical analysis and presentation of models using MLE.