contacts . discussions . jobs

Statistical significance in testing...

Hi, does anyone in the community know how I can ensure the results I get from running a test between a campaign cell and a test cell are statistically significant. I obviously want to report that any variation between the results for the campaign cell and test cell are meaningful and that the campaign actually worked, ideally. Anyone out there good at testing and statistics that can fill me in? Cheers JP

▶ Reply to This

Replies to This Discussion

Permalink Reply by Alec Saiko on February 3, 2010 at 10:52am: Usually the larger the sample size the more confident you are that your test statistic truly reflects the population and decreases your confidence interval (e.g. if 70% of your sample picks a result with a confidence interval of 5% it means you can be 'confidence level %' sure that 65-75% of the total population would pick that answer) for a given confidence level - however, this relationship isn't linear.

Basically, what you want to prove is:
you have at least 95% of confidence that x% of the whole population (that's within ±z% of y% picked by your sample) would pick the same answer as your sample.

You can easily find sample size calculators online, as well as sample size tables.; ▶ Reply to This

Permalink Reply by John-Paul Paganini on February 3, 2010 at 7:46pm: Hi Alec, Thanks for taking the time out to reply. I shall investigate further the area of sampling and confident intervals. Prior to your reply I have been under the impression that you had to apply a statistical test in order to confirm significance. Many thanks JP; ▶ Reply to This

Permalink Reply by Alec Saiko on February 8, 2010 at 2:26pm: Exactly. You do, but that's getting deeper into statistics - type1/2 errors, etc. :); ▶ Reply to This