Share this post
FaceBook  Twitter  

Measuring and analyzing estimation error are the basis of estimation learning related activities, such as deciding whether or not an organization has an estimation problem, identifying risk factors related to project performance in software development, and, evaluating and improving estimation and uncertainty assessment methods and tools.

The most commonly used measure of estimation error is the Magnitude of Relative Error (MRE). MRE is identical to the measure called FASE in other branches of forecasting research. The mean MRE (MMRE) is often used to average estimation error for multiple observations. It is not unproblematic to use MMRE as a measure of estimation accuracy, and several other measures, such as PRED and MER, is sometimes used. However, all estimation error measures have shortcomings. Hence, the measure that should be used in any given case depends on the context

Many factors can affect the measured estimation error. Measuring estimation error without a clear understanding of which factors contributed most to the estimation error, e.g., without an understanding of whether a high estimation error is caused by the factor “low estimation ability” or “high estimation complexity”, is rarely meaningful.

Papers that report estimation surveys. (Links not currently supported.)

Papers on estimation error measures and analysis. (Links not currently supported.)

See also Evaluation Methods in Principles of Forecasting: A Handbook for Researchers and Practitioners.