Call Us: US - +1 845 478 5244 | UK - +44 20 7193 7850 | AUS - +61 2 8005 4826

evaluation from a DoD perspective.

Analysis and Evaluation RESOURCES I PRIMT I HELP

Review of Some Basic Statistics Concepts, Cont.

Probability is a measure of how likely it is that an event will occur expressed as a percentage. Example: 100% ( 1.0) =event always occurs ; while 0% =event never occurs. Most probabilities of test interest fall somewhere between these two extremes.

Frequency Distribution: is a listing of the values that a variable takes in a sample from a larger population sample. T ypically these values are plotted in some manner. Common ones include:

Norm al Distribution: encountered for many naturally-occurring phenomena. Often called the “bell curve” because of the shape of its probability density curve.

Binomial Dist ribution: encountered when outcomes are measured in terms of event success or failure with only two possible outcomes (e.g., in testing, SRAW launch success or failure ) .

Exponent ial Dist ribution: is often used to model the time between independent events that happen at a constant average rate . For example, measures of success or failures over time/ distance/ cycles/ etc. In testing, it is used heavily for reliability calculations.

Confidence Level is the probability that an interval (example: SRAW maximum range = 800 ± 50 meters) will contain some percentage of the total population. For example, in this case a value of 80% means that we are 80% confident that the test data (SRAW sample value) accurately represents that the SRAW will have a maximum range between 750 and 850 meters.

….rl I Page6of 24 ~ Back Next

TST102 Fundamentals of Test Evaluation

Lesson 19 – Analysis and Evaluation RESOURCES I PRIMT I HELP

Sample Size and Confidence Levels

“Sample size” refers the number of times a specific test event is run . I t is a critical parameter in the testing process. Obviously, the more times a specific test event is performed, the higher the confidence level will likely be in terms of accuracy when test results are evaluated.

However, while increasing test sample sizes lowers the risk of inaccurate evaluations, it also directly increases testing costs and lengthens schedules. There is a point of decreasing returns where increasing sample size does not appreciably reduce risks . Part of good test planning involves using statistics to aid in the selection of an appropriate sample size for later testing that achieves a reasonable balance between evaluation accuracy and testing affordability.

Variation in test result data is a natural consequence the testing process. Excessive variation in test results, especially “outliers” (data far outside the expected curve of results ), indicates potential system performance anomalies (design flaws) or a poorly run test event.

The smaller the variance in the test data, the more confident we can be in the conclusions .