Michael's Wiki

In order to improve an objective function or energy function, want to optimize parameters used to produce it.

An easy problem is convex, where difficult ones are non-convex. In high-dimensions, heuristics are required in order to handle non-convex problems.

Common Heuristics

Rescale the dimensions to use the same units e.g. from 0 to 1 or -1 to 1. This will treat the dimensions or features equally, and may not be appropriate. This should be guided by some domain-specific knowledge of the features.

Common Optimization Methods