The group Testing consists of three modules: Power and Sample Size, Tests and Contingency Table.

Modules in the group Power and Sample Size compute power of a test, required sample size an minimal difference of parameters that can be detected by the test. The tests support both normal and binomial distribution. Inputs are significance level (or type I error) α, type of the test (one-sided or two-sided and theoretical (expected, specified) distribution parameter value. This parameter is the mean value for normal distribution, or probability for binomial distribution. Further, it is necessary to specify two of the following three numbers: Sample size, expected sample statistic and the power of the test 1 – β (β is the type II error).

Available tests:

• Normal distribution with 1 sample
• Normal distribution with 2 samples
• Binomial test for 1 ratio
• Binomial test for N ratios
• Multinomial tests

Testing - Pdf manual

Example:
For testing equality of arithmetic average x and a given value μ we use the Student t-test, where the test criterion is T = |x – μ|/s and the critical value Tα = t1–α/2(N – 1) is 100(1–α)% quantile of the Student distribution. The two curves on the above figure illustrate the densities of distribution of the test criterion value for the cases when H0 holds (g(t|H0)) and when H0 does not hold (g(t|HA)). Two types of a mistake may happen:
The type I error, when we mistakenly reject H0, despite the fact that H0 is true. This will happen, when we happen to select the data from population (or measure items from a box) that all have untypically high or low value compared to other data. This will lead to too high value of T, which consenquently, compared with Tα yields refusing H0. This situation is called the type I error ane is illustrated on . Its probability is α and can be specified by user. Usualy, we set α = 0.05, or 5%.

Type I error, H0 is rejected based on 4 unlucky measurements, though in fact H0 holds.

Similarly, we can think of type II error, when we accept H0, though it does not hold. Probability (or risk) of this situation is β. Obviously, number of data N, α, β, and difference between real and estimated parameter ∆x are interdependent. When we want for example to have low both α and β, we have to take more data. When there is big ∆x, we need less data. When we have available only small data set and expect small ∆x, we will obtain lower „reliability“ of the test in term of high α and β, etc. All methods of Power and Sample Size have both one-sided and two-sided option. One-sided option means, that we are testing only „bigger“ or only „less“, and we don’t take into account the other possibility. By two-sided test we do not distinguish between „bigger“ or „less“. One-sided option tests always x > μ in one-sample normal tests, or x2 > x1 in two-sample normal and PA > P0 in one-sample binomial proportion tests or P2 > P1 in two-sample binomial proportion tests.

The module Power and Sample Size can answer three types of questions:
- What would be the least sample size to prove the given difference between a hypothesized statistic (sample average or proportion) and a given number (or between two statistics) at a given risks α, β;
- What is the least difference difference between a hypothesized statistic (sample average or proportion) and a given number (or between two statistics) that could be proved by the test at a given sample size (or sizes) and at a given risks α, β;
- What would be the power 1 – β of a test that will prove a given difference between hypothesized statistic (sample average or proportion) and a given number (or between two statistics) at a given sample size (or sizes) and at a given risk of the type I error α.