Distributed Computation using ADE

Uses the Enterprise edition and features new Analytica 4.6

In this application, we use the COM Integration feature (introduced in Analytica 4.6) to call multiple instances of ADE (Analytica Decision Engine), evaluating the same model on both the same multi-core computer and a different computer. By doing this, we are able to take advantage of both multiple cores and multiple computers to speed up the computation on a Monte Carlo simulation. We applied this to a large model developed for and used by the North West Power Council for planning future electric grid capacity. By running 16 ADE instances on 2 computers with 8 cores each, a 10-fold speed-up was measured.

The above graph shows the elapsed time required to complete the full Monte Carlo simulation as a function of the number of parallel ADE instances utilized to carry out the simulation.

This page describes how this is done. It provides an real-life example of how to call COM objects from Analytica. You may find this useful even if you are using COM to do something much different than this, or you might be motivated to use ADE in the same manner described here.

Hardware and Licenses needed

This project uses:

• Two high-end computers running Windows Server 2012. Each server has 8 cores and more than enough RAM to hold 8 instances of the model. (The model itself evaluates in less than 1GB of RAM, each server has 16GB).
• A high-speed network connecting the two computers.

  Note: Any edition of Windows except for Windows 7+ Home edition should work here.

• One edition of Analytica Enterprise or better, release 4.6 or later.
• Two ADE licenses, one for each computer.

Distributing the computation

The master model runs in Analytica. The primary output of the model is computed by Monte Carlo simulation, usually with a sample size of 750. When run sequentially (in a single instance of Analytica), the computation takes about 50 seconds to complete. The model repeats this calculation repeatedly inside an optimization, which typically runs for 8 hours.

When the computation is distributed, the model uses COMCreateObject to instantiate up to 8 instances of ADE on each computer. Each instance loads the model, the inputs are synchronized, and each instance runs the Monte Carlo simulation using Ceil( 750/N ) samples, where N is the number of ADE instances. The ADE instances perform this computation concurrently. The sample for the final result is copied back to the master model and concatenated to get the full 750 samples.

The remainder of this page takes you through the steps and Analytica code in detail.

Specifying how the computation is distributed

To begin, we decide how many computers and how many cores the computation will be distributed over, and how that will be done, and encode that information in the model. This logic is simply deciding how the calculation shall be divided, but using COM yet.

An index contains the names of the computers where ADE can be run:

Index Host_name := ['localhost','NWDistComp2']

The model was arranged so that the user can select how many parallel ADE instances to use. The first 8 instances are "dispatched" to the first host, the next 8 to the second host, etc. The code for deciding how many instances to assign to each computer is as follows:

Variable Cores_to_use := 16    { A user input }
Variable Max_cores_per_host := Table(Host_name)(8,8)
Variable Num_ADE_Clients := Dispatch( Cores_to_use, Max_cores_per_host, Host_name )

When evaluated, Num_ADE_Clients is an array indexed by Host_name with a number showing how many ADE instances will run on each computer. With Cores_to_use set to 16, this will be 8 and 8.

This next index will index the ADE instances after we instantiate them.

Index ADE_instance := 1..Cores_to_use

And this array specifies which computer each instance of ADE will run on:

Variable Host_of_ADE_instance := StepInterp( Cumulate(Number_of_ADE_client,Distributed_Host_Nam), Distributed_Host_Nam, ADE_Instance, Distributed_Host_Nam )

Host_of_ADE_instance in an array indexed by ADE_instance where the first 8 cells are 'localhost' and the second eight cells are 'NWDistComp2'. So if you want to know where the 7th instance of ADE is running, you can look it up directly in this array.

The next variable specifies how many Monte Carlo samples each instance of ADE should run.

Variable Client_sample_size := Ceil( Size(Run) / Cores_to_use )

Notice that the master model will have the full sample size (e.g., 750), whereas each ADE client will use a smaller sample size. The build-in Run index will be used by the master model to store the full sample, but as we pull the data in from each client, the DC_Run index will correspond to the shorter index used by each client.

Index DC_Run := 1..Client_sample_size

The following mapping from each client sample into the full run index makes it easy to transform between the master and clients:

Variable Run_to_ADE_map := Min([ sampleSize, (ADE_instance-1)*Client_sample_size + DC_Run ])