Hi, does anyone in the community know how I can ensure the results I get from running a test between a campaign cell and a test cell are statistically significant. I obviously want to report that any variation between the results for the campaign cell and test cell are meaningful and that the campaign actually worked, ideally. Anyone out there good at testing and statistics that can fill me in? Cheers JP
Usually the larger the sample size the more confident you are that your test statistic truly reflects the population and decreases your confidence interval (e.g. if 70% of your sample picks a result with a confidence interval of 5% it means you can be 'confidence level %' sure that 65-75% of the total population would pick that answer) for a given confidence level - however, this relationship isn't linear.
Basically, what you want to prove is:
you have at least 95% of confidence that x% of the whole population (that's within ±z% of y% picked by your sample) would pick the same answer as your sample.
You can easily find sample size calculators online, as well as sample size tables.
Hi Alec, Thanks for taking the time out to reply. I shall investigate further the area of sampling and confident intervals. Prior to your reply I have been under the impression that you had to apply a statistical test in order to confirm significance. Many thanks JP