22

Feb

Optimizing a multivariable function parameters using a random method, Genetic Algorithm and Simulated Annealing in R

Say that you are implementing a non-linear regression analysis, which is shortly described by wikipedia as:

**“In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables.”**

For the training set, we have the following:

And the function to optimize the parameters is:

Which leads us to the following equality:

In other words, we want to optimize the value of theta in order to minimize the sum of the error among y and predicted.y: Given theta (each parameter ` a0,...,a3`

has a range from 0 to 15):

And the error function:

Finally, the goal function:

**In other words, the goal function searches for the value of theta that minimizes the error.**

This is the scatter plot of the training set:

Here is a result plot using the genetic algorithm:

You can download the code here

Tags: Machine Learning, Optimization, R