Grokbase Groups R r-help May 2006
FAQ
Hi,

I am using lmer (from the package lme4) to predict a binary response
variable (REL) from a bunch of fixed effects and two random effects
(Speaker_ID and NPhead_lemma):

fit <- lmer(REL ~
SPEAKER_GENDER +
log(SPEECHRATE) +
SQSPEECHRATE +
.....
+ (1|Speaker_ID) + (1|NPhead_lemma),
family="binomial",
data=data.lmer,
method="Laplace", model=T, x=T)

I would like to get classification accuracies for the derived model,
or even do some evaluation (cross-validation), but I cannot find a way
to get to the predicted values. As far as I can tell no predict method
has been implemented for lmer, right? I also was trying to figure out
whether lmer stores the predicted values somewhere (I read the
documentation and implementation summary,
/library/lme4/doc/Implementation.pdf).

Many thanks for your help and my apologies if I overlooked something
really simple.

Florian

--
T. Florian Jaeger
Ph.D. student
Linguistics Department,
P: +1 (650) 725 2323
F: +1 (650) 723 5666
U: http://www.stanford.edu/~tiflo/

Search Discussions

  • Spencer Graves at May 7, 2006 at 3:07 pm
    Checking 'help("lmer")' leads me also to 'help("lmer-class")', both
    of which are helpful. Unfortunately, I know of no general "predict"
    method that's available for an object of class 'lmer'. A preliminary
    function of that nature is as follows:


    predict.lmerBin <- function(object, X){
    # object has class "lmer"
    # X = model matrix with columns
    # matching object at X
    if(missing(X))
    X <- object at X
    #
    b <- fixef(object)
    X %*% b
    }


    ## To use this, you need to know how to use
    # 'preduct.glm'.

    # Example using data(Contraception), discussed in
    (mlmR <- vignette("MlmSoftRev")) # opens in Adobe Acrobat
    # To open a script file companion to vignette("MlmSoftRev")
    #edit(mlmR) # with Rgui
    #Stangle(mlmR$file) # with ESS

    fitBin <- lmer(use ~ urban+age+livch+(1|district),
    Contraception, binomial)
    predict.lmerBin(fitBin)

    Does this answer your question?
    Spencer Graves
    p.s. If your example had been simple and self-contained, it would have
    been to reply, because I could copy a few lines of R code from your
    email into R, tested a few ideas, and craft a reply in a very few
    minutes, if not seconds. Without that, crafting a sensible reply takes
    more time, partly because it's less clear what you need to know to move
    to the next step, and partly because after I guess what you are really
    asking, I must next hunt for a suitable example.

    T. Florian Jaeger wrote:
    Hi,

    I am using lmer (from the package lme4) to predict a binary response
    variable (REL) from a bunch of fixed effects and two random effects
    (Speaker_ID and NPhead_lemma):

    fit <- lmer(REL ~
    SPEAKER_GENDER +
    log(SPEECHRATE) +
    SQSPEECHRATE +
    .....
    + (1|Speaker_ID) + (1|NPhead_lemma),
    family="binomial",
    data=data.lmer,
    method="Laplace", model=T, x=T)

    I would like to get classification accuracies for the derived model,
    or even do some evaluation (cross-validation), but I cannot find a way
    to get to the predicted values. As far as I can tell no predict method
    has been implemented for lmer, right? I also was trying to figure out
    whether lmer stores the predicted values somewhere (I read the
    documentation and implementation summary,
    /library/lme4/doc/Implementation.pdf).

    Many thanks for your help and my apologies if I overlooked something
    really simple.

    Florian

    --
    T. Florian Jaeger
    Ph.D. student
    Linguistics Department,
    P: +1 (650) 725 2323
    F: +1 (650) 723 5666
    U: http://www.stanford.edu/~tiflo/

    ______________________________________________
    R-help at stat.math.ethz.ch mailing list
    https://stat.ethz.ch/mailman/listinfo/r-help
    PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
  • T. Florian Jaeger at May 7, 2006 at 11:53 pm
    An embedded and charset-unspecified text was scrubbed...
    Name: not available
    Url: https://stat.ethz.ch/pipermail/r-help/attachments/20060507/d9e36595/attachment.pl
  • Spencer Graves at May 11, 2006 at 5:01 am
    <see inline>

    T. Florian Jaeger wrote:
    Spencer,

    Thanks a lot (and sorry that I wasn't more precise). It seems to work. I
    assume this gives me the predicted log-odds for each case?
    SG: Correct.

    So, I could
    get the classification accuracy as follows? (where data.lmer is my data;
    REL is the outcome variable)

    data.lmer$lo.fit.lmer <- predict.lmerBin(fit)
    pred.fit.lmer <- exp(data.lmer$lo.fit.lmer) /
    (exp(data.lmer$lo.fit.lmer) + 1) > 0.5
    prop.table(table(pred.fit.lmer==data.lmer$REL)) # overall accuracy
    prop.table(table(data.lmer$REL, pred.fit.lmer),1) # accuracy by
    actual outcome
    SG: This looks plausible, but I don't do enough "classification" work
    to feel comfortable commenting further on this.
    Also, predict.lmerBin doesn't consider the random effects? I was
    thinking of something like the following (there are two random effects
    RA and RB, which are already coded as a factor with levels 1...k and
    1...l in the same order that you get with ranef( lmer.object); all cases
    in data don't have missing values):

    fit <- lmer(dependent ~ fixed1 + (1|RA) + (1|RB), data=data)
    lo.fit <- predict.lmerBin(fit) # predicted log-odds
    ransp <- unlist(ranef(fit)["RA"])
    rannp <- unlist(ranef(fit)["RB"])

    for (i in 1:length(lo.fit)) {
    lo.fit[i]<- lo.fit[i] + ransp[data$RA[i]] + rannp[data$RB[i]]
    }
    SG: This looks plausible, but since you still have not given me a
    self-contained example, I can't comment further. Have you considered
    'coef(fit)'? that might also help you.
    I am sure that's very unelegant, but I hope it does the right thing? It
    looks ok in some tests I ran.

    cheers,

    florian

    On 5/7/06, *Spencer Graves* <spencer.graves at pdf.com
    wrote:

    Checking 'help("lmer")' leads me also to
    'help("lmer-class")', both
    of which are helpful. Unfortunately, I know of no general "predict"
    method that's available for an object of class 'lmer'. A preliminary
    function of that nature is as follows:


    predict.lmerBin <- function(object, X){
    # object has class "lmer"
    # X = model matrix with columns
    # matching object at X
    if(missing(X))
    X <- object at X
    #
    b <- fixef(object)
    X %*% b
    }


    ## To use this, you need to know how to use
    # 'preduct.glm'.

    # Example using data(Contraception), discussed in
    (mlmR <- vignette("MlmSoftRev")) # opens in Adobe Acrobat
    # To open a script file companion to vignette("MlmSoftRev")
    #edit(mlmR) # with Rgui
    #Stangle(mlmR$file) # with ESS

    fitBin <- lmer(use ~ urban+age+livch+(1|district),
    Contraception, binomial)
    predict.lmerBin(fitBin)

    Does this answer your question?
    Spencer Graves
    p.s. If your example had been simple and self-contained, it would have
    been to reply, because I could copy a few lines of R code from your
    email into R, tested a few ideas, and craft a reply in a very few
    minutes, if not seconds. Without that, crafting a sensible reply takes
    more time, partly because it's less clear what you need to know to move
    to the next step, and partly because after I guess what you are really
    asking, I must next hunt for a suitable example.

    T. Florian Jaeger wrote:
    Hi,

    I am using lmer (from the package lme4) to predict a binary response
    variable (REL) from a bunch of fixed effects and two random effects
    (Speaker_ID and NPhead_lemma):

    fit <- lmer(REL ~
    SPEAKER_GENDER +
    log(SPEECHRATE) +
    SQSPEECHRATE +
    .....
    + (1|Speaker_ID) + (1|NPhead_lemma),
    family="binomial",
    data=data.lmer,
    method="Laplace", model=T, x=T)

    I would like to get classification accuracies for the derived model,
    or even do some evaluation (cross-validation), but I cannot find a way
    to get to the predicted values. As far as I can tell no predict method
    has been implemented for lmer, right? I also was trying to figure out
    whether lmer stores the predicted values somewhere (I read the
    documentation and implementation summary,
    /library/lme4/doc/Implementation.pdf).

    Many thanks for your help and my apologies if I overlooked something
    really simple.

    Florian

    --
    T. Florian Jaeger
    Ph.D. student
    Linguistics Department,
    P: +1 (650) 725 2323
    F: +1 (650) 723 5666
    U: http://www.stanford.edu/~tiflo/

    ______________________________________________
    R-help at stat.math.ethz.ch <mailto:r-help@stat.math.ethz.ch>
    mailing list
    https://stat.ethz.ch/mailman/listinfo/r-help
    PLEASE do read the posting guide!
    http://www.R-project.org/posting-guide.html




    --
    T. Florian Jaeger
    Ph.D. student
    Linguistics Department,
    P: +1 (650) 725 2323
    F: +1 (650) 723 5666
    U: http://www.stanford.edu/~tiflo/
  • S. Messing at Nov 5, 2009 at 10:16 pm
    I had to deal with this issue yesterday, and I'm guessing many who use lme4
    have this same issue because of the lack of a predict.lmer function. It
    seems to me that one would need to plug the X %*% b output from the
    predict.lmerBin function below into the logistic distribution to get the
    actual predicted probabilities. In other words:

    predict.lmerBin <- function(object, X){
    if(missing(X))
    X <- object at X
    b <- fixef(object)
    plogis(X %*% b)
    }

    The example below demonstrates the equivalence between plogis(X %*% b) and
    predict.glm(..., type = "response",...) :

    ldose <- rep(0:5, 2)
    numdead <- c(1, 4, 9, 13, 18, 20, 0, 2, 6, 10, 12, 16)
    sex <- rep(c(1, 2), c(6, 6))
    SF <- cbind(numdead, numalive -numdead)
    budworm.lg <- glm(SF ~ sex*ldose, family=binomial)
    summary(budworm.lg)
    ld <- seq(0, 5, 0.1)

    preds = predict(budworm.lg, newdata= data.frame(sex=1, ldose=ld),
    type = "response")
    X = as.matrix(data.frame(int=1,sex=1, ldose=ld, sex.ldose=1*ld))#, sex.ldose
    = 2*ld))
    b=coef(budworm.lg)
    plogis(X%*%b) == preds

    If anyone can see a problem with the above approach, please let me know as I
    am using this for a paper.

    -Solomon


    Spencer Graves wrote:
    Checking 'help("lmer")' leads me also to 'help("lmer-class")', both
    of which are helpful. Unfortunately, I know of no general "predict"
    method that's available for an object of class 'lmer'. A preliminary
    function of that nature is as follows:


    predict.lmerBin <- function(object, X){
    # object has class "lmer"
    # X = model matrix with columns
    # matching object at X
    if(missing(X))
    X <- object at X
    #
    b <- fixef(object)
    X %*% b
    }


    ## To use this, you need to know how to use
    # 'preduct.glm'.

    # Example using data(Contraception), discussed in
    (mlmR <- vignette("MlmSoftRev")) # opens in Adobe Acrobat
    # To open a script file companion to vignette("MlmSoftRev")
    #edit(mlmR) # with Rgui
    #Stangle(mlmR$file) # with ESS

    fitBin <- lmer(use ~ urban+age+livch+(1|district),
    Contraception, binomial)
    predict.lmerBin(fitBin)

    Does this answer your question?
    Spencer Graves
    p.s. If your example had been simple and self-contained, it would have
    been to reply, because I could copy a few lines of R code from your
    email into R, tested a few ideas, and craft a reply in a very few
    minutes, if not seconds. Without that, crafting a sensible reply takes
    more time, partly because it's less clear what you need to know to move
    to the next step, and partly because after I guess what you are really
    asking, I must next hunt for a suitable example.

    T. Florian Jaeger wrote:
    Hi,

    I am using lmer (from the package lme4) to predict a binary response
    variable (REL) from a bunch of fixed effects and two random effects
    (Speaker_ID and NPhead_lemma):

    fit <- lmer(REL ~
    SPEAKER_GENDER +
    log(SPEECHRATE) +
    SQSPEECHRATE +
    .....
    + (1|Speaker_ID) + (1|NPhead_lemma),
    family="binomial",
    dataÚta.lmer,
    method="Laplace", model=T, x=T)

    I would like to get classification accuracies for the derived model,
    or even do some evaluation (cross-validation), but I cannot find a way
    to get to the predicted values. As far as I can tell no predict method
    has been implemented for lmer, right? I also was trying to figure out
    whether lmer stores the predicted values somewhere (I read the
    documentation and implementation summary,
    /library/lme4/doc/Implementation.pdf).

    Many thanks for your help and my apologies if I overlooked something
    really simple.

    Florian

    --
    T. Florian Jaeger
    Ph.D. student
    Linguistics Department,
    P: +1 (650) 725 2323
    F: +1 (650) 723 5666
    U: http://www.stanford.edu/~tiflo/

    ______________________________________________
    R-help at stat.math.ethz.ch mailing list
    https://stat.ethz.ch/mailman/listinfo/r-help
    PLEASE do read the posting guide!
    http://www.R-project.org/posting-guide.html
    ______________________________________________
    R-help at stat.math.ethz.ch mailing list
    https://stat.ethz.ch/mailman/listinfo/r-help
    PLEASE do read the posting guide!
    http://www.R-project.org/posting-guide.html
    --
    View this message in context: http://old.nabble.com/qu%3A-predict-with-lmer-%28lme4%29-or-other-ways-to-get-classification-accuracy-tp4203433p26223153.html
    Sent from the R help mailing list archive at Nabble.com.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupr-help @
categoriesr
postedMay 3, '06 at 2:52a
activeNov 5, '09 at 10:16p
posts5
users3
websiter-project.org
irc#r

People

Translate

site design / logo © 2022 Grokbase