### How can I control classifier parameter optimization?

PRTools has some built-in possibilities to optimize parameters used for defining classifiers. For instance, the quadratic classifier based on the assumption of normal distributions, `qdc`, has some parameters to control the regularization. They adapt the estimated covariance matrix into the direction of a diagonal matrix based on the feature variances, or the average feature variance. A call like

`A = qdc(A,alf)`

regularizes the covariance matrix by a weighted average with a diagonal matrix. The weights are `1-alf` and `alf`. Users may try a few values like 1e-2 and 1e-3, or, instead, set `alf` = `NaN`. In this case PRTools optimizes `alf` by at most 20 calls to the Matlab optimizer `fminbnd`, on an interval set inside `qdc`. The criterion is a 5-fold cross-validation error, see below for details. A soft error criterion is used while a crisp error causes problems as minor changes in `alf` may not cause a difference in the error.This procedure may slow down training by a factor of 100 (5 x 20).

It is also possible to optimize more than one parameter simultaneously. In this case not a full grid search is performed, but the parameters are optimized one after the other in a on order specified by the user. Defaults are used for parameters not yet optimized.

This procedure is controlled by the PRTools routine `regoptc`, called when one of the parameters is `NaN`. This might not be programmed in all functions. Moreover, users may want to control some settings. This can be solved by calling `regoptc` externally. Here is an example based on the PRTools support vector classifier `svc`. We will simultaneously optimize the degree of the polynomial kernel and the trade-off parameter C.

```A = gendatb;                  % dataset pars = {'p',NaN,NaN};         % initial parameters. NaNs will be optimized defs = {'p',1,1};             % set defaults for all classifier parameters parmin_max = [0, 0; 1,10; 1e-2, 1e2]; % set interval for every parameter par_order = [2 3 1];          % optimization order of parameters realint = [1 0 1];            % 2nd par is integer, others real testfunc = testc([],'soft');  % test function W = regoptc(a,'svc',pars,defs,par_order,parmin_max,testfunc,realint); delfigs; scatterd(A); plotc(W) getopt_pars                   % show final parameter values```

This example usually finds a 3rd order polynomial kernel and a trade-off parameter C that is somewhat larger than 1. The test function should be a mapping that can be called like `E = A*W*testfunc` for a computation of the criterion to be minimized.

The call to regoptc is somewhat inconsistent. It requires that the classifier function is given by a string and the test function by a mapping. This is needed as the regoptc needs to know what parameters have to be optimized. Parameter values to be tested are determined by `fminbnd` (or `nfminbnd` for integer parameters). The function to be minimized is an error estimated by cross validation using the PRTools routine `prcrossval`. The user specified mapping `testfunc` (see the above example) is used inside this routine. The number of folds and the number of repetitions are determined by the globals  `REGOPT_NFOLDS` and `REGOPT_REPS`, preset by 5 and 1. The maximum number of iterations for `fminbnd` and `nfminbnd` is set `by REGOPT_ITERMAX`, preset by 20. Users may change these values by `prglobal`.