# significance_analysis {bandit}

### Description

A convenience function to perform overall proportion comparison using prop.test, before doing pairwise comparisons, to see what outcomes seem to be better than others.

### Usage

significance_analysis(x, n)

### Arguments

- x
- as in prop.test, a vector of the number of successes
- n
- as in prop.test, a vector of the number of trials

### Values

a data frame with the following columns:

- successes
- x
- totals
- n
- estimated_proportion
- x/n
- lower
- 0.95 confidence interval on the estimated amount by which this alternative outperforms the next-lower alternative
- upper
- 0.95 confidence interval on the estimated amount by which this alternative outperforms the next-lower alternative
- significance
- p-value for the test that this alternative outperforms the next-lower alternative
- order
- order, by highest success proportion
- best
- 1 if it is part of the 'highest performing group' -- those groups which were not significantly different from the best group
- p_best
- Bayesian posterior probability that this alternative is the best binomial bandit

### Note

This is intended for use in A/B split testing -- so sizes of n should be roughly equal. Also, note that alternatives which have the same rank are grouped together for analysis with the 'next-lower' alternative, so you may want to check to see if ranks are equal.

### See Also

### Examples

x = c(10,20,30,50) n = c(100,102,120,130) sa = significance_analysis(x,n) sa[rev(order(sa$estimated_proportion)), ] x = c(37,41,30,43,39,30,31,35,50,30) n = rep(50, length(x)) sa = significance_analysis(x,n) sa[rev(order(sa$estimated_proportion)), ] x = c(37,41,30,43,39,30,31,37,50,30) n = rep(50, length(x)) sa = significance_analysis(x,n) sa[rev(order(sa$estimated_proportion)), ]

Documentation reproduced from package bandit, version 0.4.2. License: GPL-3