Different regression criteria exhibit different properties when used for fitting purposes. In this paper the properties of different regression criteria are described in detail. The underlying error structure and the link between the regression criterion and the application area is analysed. Examples in different modeling areas are provided. Given are an example of device modeling, system-level modeling, and the temperature decay of a biomedical sensor chip. The examples show that there doesn't exist an ideal regression criterion. The most appropriate regression criterion depends on the constitution of the total error. This makes fitting a multiple step process. Only after some trial fittings one is able to determine the most appropriate regression criterion for a given application.