R 4.x.x new S3 methods dispatch and local environments

Introduction

R 4.0.0 and subsequent 4.x.x versions now have a new way to lookup for S3 methods associated with generic functions. This has posed some troubles to people as me who like to store their own generic functions and related methods in local environments, maybe via sourcing scripts with sys.source, populating a particular environment and then attaching such environment to the search list. Others, use the function local to define their own methods, populate the environment and then attach it to make all the methods available in the search list. These two methods have in common that they evaluate functions definitions in a user specified environment, providing some sort of package-like set of functions conveniently living outside the .GlobalEnv.

The problem is that as of version 4.0.0, S3 method lookup now by default skips the elements of the search path between the global and base environments. The final result is that either sourcing on a specific environment or defining new S3 methods in local environments will not work as usual, as in R 3.x.x versions, say. Indeed, if you rely on such technique to gain control over your session, you will find messages such as

Error in UseMethod("foo") : 
  no applicable method for 'foo' applied to an object of class "character"

for a generic «foo» and a S3 method «foo.character«. I particularly use sys.source in every project and populate it with my own set of functions using the .Rprofile file. In this way, I start a clean R session and the set of functions I depend on are always available. Note that sys.source is given as a natural way to do this as can be seen in its documentation entry. This is convenient because no NAMESPACE technology is required as in package building. However, as of R 4.0.0 apparently new S3 methods need to be registered properly via the .S3method or registerS3method functions. This complicates matters as such registry should be made «after the top level environment … of the calling environment.«

The new .S3method

It is worth noting that using .S3method works properly with new S3 methods defined in local environments for preexisting generics:

myenv <- new.env()
myenv$print.myclass <- local(function(x, ...) print(formatC(x$y, format="f", digits=5)), new.env(myenv))
.S3method("print", "myclass", myenv$print.myclass)
attach(myenv)
rm(list=ls())
myfun <- function(y) {
   out <- list(y=y)
   class(out) <- "myclass"
   return(out)
}
myfun(1:4)

# [1] "1.00000" "2.00000" "3.00000" "4.00000

which prints what is intended.

However, you might want to define your own new generic:

myenv <- new.env()
myprint <- local(function(x, ...) UseMethod("myprint"), myenv)
myenv$myprint <- myprint
myenv$myprint.myclass <- local(function(x, ...) print(formatC(x$y, format="f", digits=5)), myenv)
.S3method("myprint", "myclass", myenv$myprint.myclass)
attach(myenv)
rm(list = ls())
myfun <- function(y) {
   out <- list(y=y)
   class(out) <- "myclass"
   return(out)
}
myprint(myfun(1:4))

Error in UseMethod("myprint") :
  no applicable method for 'myprint' applied to an object of class "myclass"

(these are adaptations from https://r.789695.n4.nabble.com/S3-method-dispatch-for-methods-in-local-environments-td4763154.html.)

Solution 1. Defining new generics on the go

This is basically just to properly define new generics and S3 methods in the .GlobalEnv userspace.

myenv <- new.env()
myprint <- function(x, ...) UseMethod("myprint")
myenv$myprint <- myprint
myenv$myprint.myclass <- function(x, ...) print(formatC(x$y, format="f", digits=5))
.S3method("myprint", "myclass", myenv$myprint.myclass)
attach(myenv) #attach myenv to the search list
rm(list = ls()) #removes myenv from .Globalenv
myfun <- function(y) {
   out <- list(y=y)
   class(out) <- "myclass"
   return(out)
}
myprint(myfun(1:4))

# [1] "1.00000" "2.00000" "3.00000" "4.00000

Solution 2. Sourcing new generics and S3 methods

Say you have an R script, myprint.R say, with the new generic myprint and a new S3 method as above; be sure you use .S3method:

myprint <- function(x, ...) UseMethod("myprint")
myprint.myclass <- function(x, ...) print(formatC(x$y, format="f", digits=5))
.S3method("myprint", "myclass", myprint.myclass)

You might now source it in at least two ways that work just fine (be sure to substitute path_to_script/ appropriately:

First way: S3 registry table in base.env

rm(list=ls())
sys.source("path_to_script/myprint.R")
ls() #you'll get character(0)
myfun <- function(y) {
   out <- list(y=y)
   class(out) <- "myclass"
   return(out)
}
myprint(myfun(1:4))

This method has probably the disadvantage that the base.env is populated with your new generics and methods, but they are properly registered.

Second way: S3 registry table in .GlobalEnv, but functions available in an attached environment

rm(list=ls())
myfunctions <- new.env()
attach(myfunctions)
rm(myfunctions)
sys.source("path_to_script/myprint.R", envir = .GlobalEnv)
ls()
for(i in ls(.GlobalEnv)){
    assign(i, get(i), envir = as.environment("myfunctions"))
}
rm(list=ls())
ls() #nothing in .GlobalEnv
ls(as.environment("myfunctions")) #generics and methods in "myfunctions"
myfun <- function(y) {
   out <- list(y=y)
   class(out) <- "myclass"
   return(out)
}
myprint(myfun(1:4))
# [1] "1.00000" "2.00000" "3.00000" "4.00000

This method has the advantage that the attached environment, in this case myfunctions is populated with your new generics and methods, and they are properly registered, this time in .GlobalEnv.

ls(, all.names = T)
[1] ".__S3MethodsTable__." "myfun"

The key here is the .__S3MethodsTable__. environment which handles the S3 registry. In the first way, generics and methods are stored in the base.env and here is also where the .__S3MethodsTable__. will also be present.

Former method via sys.source (R 3.x.x)

I used to simply source.sys a script file or a bunch of them into an environment and then simply attach it to the search session path; all this in the .Rprofile file:

myfunctions <- new.env()
sys.source("path_to_script/myprint.R", envir = myfunctions)
attach(myfunctions)
rm(myfunctions)

However, this technique does not work from R 4.0.0 onward.

Conclusion

The usual way to S3 methods were dispatched is no longer possible en R 4.x.x. If one wants to create a package-like environment attached to the R session, probably by sourcing the files containing your own generics and methods, one has to first generate the S3 registry in the .GlobalEnv environment. Then one can assign the set of functions just sourced into an environment, which can then be attached to the session. The .S3method should be used in the sourced files to generate the S3 registry. This might be a bit of added bureaucracy to your project and indeed it somehow resembles building packages using NAMESPACE files. This however apparently increases efficiency. It is worth noting that using the technique of sourcing files into an environment other than .GlobalEnv is suggested already in the R documentation of previous versions.

References


  1. https://r.789695.n4.nabble.com/S3-method-dispatch-for-methods-in-local-environments-td4763154.html
  2. https://developer.r-project.org/Blog/public/2019/08/19/s3-method-lookup/
  3. https://stat.ethz.ch/pipermail/r-announce/2020/000653.html

Deja un comentario