Multithreaded evaluation

Some computations can take advantage of multiple cores on your computer to speed up to net computation time. These computations partition the task across multiple threads, where each thread can be assigned to a different core. For example, when adding two large arrays, A+B, where each array has 100,000 elements, the computation might be parallelized across four threads, with each thread handling roughly 25,000 additions each. As a result, the elapsed time to complete the full array addition might be as much as four times faster than it would have been when evaluated in single threaded mode. Because there is additional overhead required to partition a task and coordinate multiple threads, the actual speedup will end up being something less that the total number of threads.

Multithreaded evaluation uses fine-grained parallelism, meaning that certain low-level computations are parallelized. The computation is decomposed at a high-level. So when concurrent evaluation is occurring, all threads are working on the same task.

Not all operations are decomposed into parallel computation. In general, arithmetic operations are performed concurrently, but only when the arrays are large enough to justify the extra overhead of multithreading. As a rough idea, operators involving arrays with fewer than 100 elements will be evaluated on a single thread, operations involving more than 1,000 cells are likely to be performed on multiple threads (if threads are available), and between there the threshold depends on the internal array structure, specific operation, etc. Several built-in functions operate in parallel, as well as some internal bookkeeping, but again, these use multiple threads only when the arrays involved are large enough to justify the overhead. With new releases of Analytica, additional functions or operations may be enhanced to leverage multithreaded evaluation.

Requirements

Multithreaded evaluation is available in all Analytica editions except Analytica Free Edition.

In 32-bit versions of Analytica, multithreaded evaluation is disabled by default. We don't recommend enabling it in most cases, because the extra threads in multi-threaded evaluation consume memory. Since 32-bit applications can't use more than 3 or 4GB RAM, the advantage of more efficient memory use by single-threading usually outweighs the the speed-up from multithreaded evaluation.

Configuration

The configuration of multithreaded evaluation is controlled by two system variables. To change these, navigate the menus to Definition > System variables > Settings.

Max Evaluation Threads

Set this to 1 to force single threaded evaluation, or to a number greater than 1, up to the number of available cores on your computer, to allow multithreaded evaluation. Evaluation will limit the number of threads actually used to this number.

This setting is specific to your computer, and not associated with any model. It persists when you quit and restart Analytica.

If you want to prevent computations from periodically usurping all available cycles, you may opt to set this to a number less than the total number of cores on your computer. If you are running multiple instances of Analytica or ADE, each may use up to this number of threads. Also, since each thread used does require extra memory, you might want to limit the maximum number of threads when you have a small amount of RAM on your computer.

If multiple users use Analytica from the same computer, but from different accounts, each user has his own setting for Max Evaluation Threads.

Max Evaluation Threads in ADE

In ADE, the max evaluation threads does not persist from instance to instance. This is so that in a web application, one session changing it doesn't impact other user sessions. You can configure the initial value for ADE sessions in a registry setting using RegEdit:

HKEY_LOCAL_MACHINE / Software / Lumina Decision Systems / ADE /5.2

If your program want to reduce this, it can do so using, e.g.,

ade.SendCommand("MaxEvaluationThreads:1")

where ade is your CAEngine instance.

Max Model Threads

Set this to 1 to force single threaded evaluation for the current model, or to 0 to eliminate any maximum number of threads. You can set it to a number greater than 1 to enforce a maximum number of threads that will be used for evaluation by the current model. The number of threads actually used for evaluation will never exceed this number.

This setting is stored with the current model, and so will be set when you share your model with other people, but it won't impact other models.

Use this setting when you need to limit the number of threads used only for the current model. When Max Model Threads is set, the actual maximum number of threads used will be the smaller of Max Model Threads and Max Evaluation Threads.

One reason to force a model to be single threaded would be a bug were to be discovered that impacts your model with multithreaded evaluation. Another would be if you were to discover that multithreaded evaluation was resulting in a net slowdown for your model, or if you were to find that your model is fastest at some particular number of threads.

Net speed-up gains from multi-threaded evaluation

The net speed-up obtained from multi-threaded evaluation depends strongly on the particular model and computations performed. In general, models that manipulate larger arrays are more likely to derive more benefit.

Suppose that 60% of the operations in your model (as measure as a percentage of time in single-threaded evaluation) are able to take advantage of multithreaded evaluation, and these operations experience a 3-fold speedup as a result (as might be realistic with 4 cores after overhead). Then 60% * 2/3 = 40% of the total evaluation time will be saved, which results in a 66% speed-up. So even with four cores, and fairly favorable conditions, the net speed-up is less than a factor of 2.

History

Analytica 5.0 was the first release to include multithreaded evaluation.