R Programming/Optimization

Optimization (mathematics) Statistics/Numerical_Methods/Optimization optimize() is devoted to one dimensional optimization problem. optim(), nlm(), ucminf() (ucminf) can be used for multidimensional optimization problems. nlminb() for constrained optimization. quadprog, minqa, rgenoud, trust packages Some work is done to improve optimization in R. See Updating and improving optim(), Use R 2009 slidesUpdating and improving optim(), Use R 2009 slides http://www.agrocampus-ouest.fr/math/useR-2009/slides/Nash+Varadhan.pdf , the R-forge optimizer pageR-forge optimizer http://optimizer.r-forge.r-project.org/ and the corresponding packages including optimx. Numerical Methods One dimensional problem The one dimensional problem : > func <- function(x){ + return ( (x-2)^2 ) + } > (func(-2)) [1] 16 > > # plot your function using the 'curve function' > curve(func,-4,8) > > # Here is another way to plot the function > # using a grid > grid <- seq(-10,10,by=.1) > func(grid) > plot(grid,func(grid)) > > # you can find the minimum using the optimize function > optimize(f=func,interval=c(-10,10)) $minimum [1] 2 $objective [1] 0 Newton-Raphson nlm() provides a Newton algorithm. maxLik package for maximization of a likelihood function. This package includes the Newton Raphson method. newtonraphson() in the spuRs package. BFGS The BFGS method > func <- function(x){ + out <- (x[1]-2)^2 + (x[2]-1)^2 + return <- out + }> > optim(par=c(0,0), fn=func, gr = NULL, + method = c("BFGS"), + lower = -Inf, upper = Inf, + control = list(), hessian = T) > optim(par=c(0,0), fn=func, gr = NULL, + method = c("L-BFGS-B"), + lower = -Inf, upper = Inf, + control = list(), hessian = T) Conjugate gradient method Conjugate gradient method optim() with method="cg". Trust Region Method "trust" package for trust region method The Nelder-Mead simplex method The Nelder Mead Method > func <- function(x){ + out <- (x[1]-2)^2 + (x[2]-1)^2 + return <- out + } > > optim(par=c(0,0), fn=func, gr = NULL, + method = c("Nelder-Mead"), + lower = -Inf, upper = Inf, + control = list(), hessian = T) The boot package includes another simplex method Simulation methods Simulated Annealing Simulated annealing The Simulated Annealing is an algorithm which is useful to maximise non-smooth functions. It is pre implemented in optim(). > func <- function(x){ + out <- (x[1]-2)^2 + (x[2]-1)^2 + return <- out + }> > optim(par=c(0,0), fn=func, gr = NULL, + method = c("SANN"), + lower = -Inf, upper = Inf, + control = list(), hessian = T) EM Algorithm EM algorithm Genetic Algorithm Genetic algorithm rgenoud package for genetic algorithmJasjeet Sekhon homepage : http://sekhon.berkeley.edu/rgenoud/ gaoptim package for genetic algorithmgaoptim on CRAN: http://cran.r-project.org/web/packages/gaoptim/index.html ga general purpose package for optimization using genetic algorithms. It provides a flexible set of tools for implementing genetic algorithms search in both the continuous and discrete case, whether constrained or not. ga on CRAN: http://cran.r-project.org/web/packages/GA/index.html/ References Citations Sources Venables and Ripley, Chapter 16. Cameron and Trivedi, Microeconometrics, Chapter 10. Braun and Murdoch, first course in statistical programming with R (a very good reference on optimization using R), Chapter 7. Mathematics Probability Distributions Category:Newton's method