Sunday, March 4, 2012

DEoptim in Parallel

Running DEoptim in parallel has been on the development team's wishlist for awhile.  It had not been a priority though, because none of us have personally needed it.  An opportunity arose when Kris Boudt approached me about collaborating to add this functionality as part of a consultancy project for a financial services firm.

We were able to add and test the functionality within a week.  The latest revision of DEoptim on R-Forge has the capability to evaluate the objective function on multiple cores using foreach.  Very CPU-intensive problems will see speed increases in approximately linear time (less communication overhead).

I gave a short presentation (PDF) on the parallel functionality at the Saint Louis R User Group meetup in February.  A longer-running version of the code used in the presentation is on R-Forge, in the file DEoptim/sandbox/largeN_doSNOW.R (revision 86).

There are a few things to keep in mind when using the parallel functionality.  I quote from the meetup presentation:
  • Data communication between nodes can overwhelm gains from processing on multiple CPUs
    • Be careful with non-varying objects
    • Exclude them from formal function arguments
  • Copy them to nodes before optimization (clusterExport)
  • If mu and sigma were formal function arguments, they would be copied to each node for all 2037 function evaluations!
Please try it and give us feedback.  R-Forge has been undergoing major updates, so please anonymously checkout the source and build it yourself if you're unable to download the pre-built source / binaries.

6 comments:

Unknown said...

This is great work by Josh. If anyone would like to test this out before we send it to CRAN, see example code in the 'sandbox' directory in the package source, also provided by Josh.

If you want to just see it working (though this won't show any speed improvements), here is some minimal code. You will need the following packages installed: doSNOW, foreach, DEoptim (the version from R-forge, not CRAN).



require(DEoptim)

'Rosenbrock' <- function(x){
x1 <- x[1]
x2 <- x[2]
100 * (x2 - x1 * x1)^2 + (1 - x1)^2
}

set.seed(1234)

t1 <- system.time(out1 <- DEoptim(Rosenbrock, rep(-10,2), rep(10,2)))
out1$optim$iter
out1$optim$bestval

require(doSNOW)

cl <- makeSOCKcluster(2)

## do something like the lines below if necessary:
##clusterEvalQ(cl, library(PerformanceAnalytics)) # load any necessary libraries
##clusterExport(cl, list("mu", "sigma", "m3", "m4")) # copy any necessary objects

## you can monitor your processes (using, e.g., the 'top'
## command on a unix shell, and see multiple R
## sessions created

registerDoSNOW(cl) # register foreach backend
set.seed(1234)
t2 <- system.time(out2 <- DEoptim(Rosenbrock, rep(-10,2), rep(10,2)))
stopCluster(cl) # stop cluster
out2$optim$iter
out2$optim$bestval

## faster in serial
t1
t2

Unknown said...

Hi there,

Very nice job indeed. I tried the package on a very basic problem (solving the MV optimal portfolio problem with DEoptim with a bunch of assets, which is inefficient), but I did not observe time improvement for the calculation. Might well be some inefficiency in my own code.

Olivier.

Unknown said...

Here's a gist that modifies the example code in the sandbox dir so that it uses doMC for the unix folks:

https://gist.github.com/2050019

Rick said...

I was applying parallel DEoptim on my data, but it is even slower than seqential.. which makes me quite confused.

Joshua Ulrich said...

Rick,

You've probably encountered what I mentioned in the first bullet point, "Data communication between nodes can overwhelm gains from processing on multiple CPUs".

Running in parallel will not be beneficial for every problem.

Chicago said...

Thanks Josh! -Stu