On Fri, 1 Mar 2002 joern.quedenau at metanomics.de wrote:

Dear all,

I found a discrepancy while performing a power calculation for a two sample

t-test in R and S-PLUS, respectively.

For given values of sample number (5 each), sd (0.2) , significance level

(0.01), and a desired power (80%) I looked for the difference in means.

These values differ: 0.5488882 in R and 0.4322771 in S-PLUS (see dump

below).

Did I overlook any detail or confuse some parameters?

The S-PLUS routine references Fisher & van Belle. In that book the

authors use a unified approximate power calculation method that works for

a wide range of studies but is not very accurate for tiny sample sizes.

In most cases this doesn't matter because the assumptions going into a

study design aren't any more accurate, and in tiny sample sizes the power

is sensitive to the assumption that the data are Normally distributed.

The power.t.test formula uses the non-central t distribution and so will

give more accurate, lower power values for small samples.

You can see which one is correct by simulation (which is how I typically

do power calculations)

table(sapply(1:10000,function(i)

t.test(rnorm(5,0,s=0.2),rnorm(5,.5488882,s=0.2),var.equal=TRUE)$p.value)<=0.01)

FALSE TRUE

2023 7977

table(sapply(1:10000,function(i)

t.test(rnorm(5,0,s=0.2),rnorm(5,.4322771,s=0.2),var.equal=TRUE)$p.value)<=0.01)

FALSE TRUE

4526 5474

So at 0.548882 there is about 80% power, at 0.4322771 there is about 55%

power (with sampling uncertainties of about +/- 2% in each number).

It is interesting to note that a simulation shows the unequal-variance

t-test to have only 75% power at 0.5488882, indicating the sensitivity of

the power calculations at this sample size.

-thomas

Thomas Lumley Asst. Professor, Biostatistics

tlumley at u.washington.edu University of Washington, Seattle

Thanks for your help,

Joern Quedenau

Here are the commands & outputs from both tools:

R 1.4.0

power.t.test(n=5, sd=0.2, sig.level=0.01, power=0.8, type="two.sample",

alternative="two.sided")

Two-sample t test power calculation

n = 5

delta = 0.5488882

sd = 0.2

sig.level = 0.01

power = 0.8

alternative = two.sided

NOTE: n is number in *each* group

S-PLUS 2000 Professional Release 2:

normal.sample.size(n1=5, n2=5, mean=0, sd1=0.2, sd2=0.2, power=0.8,

alpha=0.01, alternative="two.sided")

mean1 sd1 mean2 sd2 delta alpha power n1 n2 prop.n2

1 0 0.2 0.4322771 0.2 0.4322771 0.01 0.8 5 5 1

------------------------------------------

Dr. Jörn Quedenau

Coordinator Data Management Bioinformatics

Metanomics GmbH & Co. KGaA

Tegeler Weg 33, D-10589 Berlin, Germany

Tel +49 30 34807 125, Fax +49 30 34807 300

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-

r-help mailing list -- Read

http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.htmlSend "info", "help", or "[un]subscribe"

(in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch

_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

Thomas Lumley Asst. Professor, Biostatistics

tlumley at u.washington.edu University of Washington, Seattle

^^^^^^^^^^^^^^^^^^^^^^^^

NOTE NEW EMAIL ADDRESS

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-

r-help mailing list -- Read

http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.htmlSend "info", "help", or "[un]subscribe"

(in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch

_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._