Looping over a model

When you want to evaluate your model over many input scenarios, Analytica's array abstraction mechanism is very convenient. You simply add indexes to your inputs and array abstraction does the rest automatically, propagating those indexes to all downstream results. However, at some point you may find yourself limited by the amount of usable memory, since all those intermediate results, for all scenarios, are cached and consuming memory.

To get around this, you may consider looping over your model (or a subset of your model), and collecting the results for all scenarios only for a selected output variable. By doing so, the intermediate steps in your model consume only the memory required for a single scenario. This technique can sometimes enable a large scale analysis that would otherwise not be possible.

This technique of looping over your model is only possible when your model does not operate over the index you are looping over. If any variable in your model operates over the index, such as the way Sum(..., I) operates over index I, or SDeviation(X) operates over the Run index, then looping is not a viable option.

The basic technique

The basic technique is demonstrated by the following example. Suppose your model has an input X, indexed only by I, and an output Y, and that the intermediate steps to no operate over the index I. Then we can compute Y by looping as follows:

 Variable Y_by_looping := 
    For xi[ ] := X do WhatIf(Y, X, xi)

The same technique can be used for large-scale Monte Carlo sampling. Suppose we have a single uncertain scalar input, U, and a result Z. We can compute the result for Z one-sample at a time using:

 Variable Z_by_looping :=
    For ui[ ] := U do WhatIf(Z, U, ui)

With this technique you need to be careful to ensure that your model does not use any statistical functions, since these operate over the Run index. You must also be careful not to switch evaluation mode. In other words, Sample(Z) must not depend on Mid(U), and Mid(Z) must not depend on Sample(Z).

Extensions

Numerous extensions to the above simple technique are often required. These are described here.

Other input dimensions

Your inputs may have indexes that your model does operate over, so you cannot loop over those indexes. For example, an input X may have four dimensions: Time, District, AssumptionSet and Scenario. Your model operates over the Time and District indexes, so you can't loop over these, but you do wish to loop over the AssumptionSet and Scenario indexes. The above approach is then altered by specifying the indexes that are to be left in-tact (i.e., Time) in the inputs as follows:

 Variable Y_by_looping := 
    For xi[Time, District] := X do WhatIf(Y, X, xi)

Alternatively, instead of being explicit about which indexes are not looped over, you can be explicit about which indexes are looped over:

 Variable Y_by_looping := 
    For s := Scenario do (
       For a := AssumptionSet do (
          WhatIf( Y, X, X[Scenario = s, AssumptionSet = a])
       )
    )

The first technique of listing variables that are not looped over is in many ways the most flexible, since the technique continue to keep memory usage low of other parametric dimensions are introduced by the user.

Multiple inputs

When you have more than one input that uses the index being looped over, the technique is extended by using nested WhatIf calls. To coordinate the "slicing" among the inputs, the easiest and most flexible technique is to introduce a User-Defined Function, and to declare the dimensions of each input that are NOT looped over in the parameter list:

  Function ComputeY(v1, v2: Array[Time, District]) :=  WhatIf(WhatIf(Y, X1, v1), X2, v2)
   
  Variable Y_by_looping := ComputeY(X1, X2)

Multiple Outputs

The key to keeping memory consumption low is to only collect the full results for a handful of output variables. If you have more than one output of interest, you want to coordinate the computation so that all are computed simultaneously with each pass. If you have a probability distribution in your model that depends on the selected inputs, and more than one output depends on that probability distribution, then you need to compute both outputs simultaneously so that they are based on the same sample. Also, by computing all outputs simultaneously, you avoid having to re-compute intermediate results each time.

To compute multiple results, you should bundle these into a record structure using references.

 Index LoopOutputs := ['Y1', 'Y2', 'Y3']
  
 Function ComputeOutputs(v1, v2: Array[Time, District]) :=
    WhatIf(WhatIf(Array(LoopOutputs, [\Y1, \Y2, \Y3]), X1, v1), X2, v2)

 Variable Y2_by_looping := ComputedBy(Y1_by_looping)
  
 Variable Y3_by_looping := ComputedBy(Y1_by_looping)
  
 Variable Y1_by_looping := 
    Var y := ComputeOutputs(x1, x2);
    Y2_by_looping := #y[LoopOutputs = 'Y2'];
    Y3_by_looping := #y[LoopOutputs = 'Y3'];
    #y[LoopOutputs = 'Y1']

Hybrid Abstraction

Analytica's vertical abstraction method of computing every variable to completion is computationally faster than looping (horizontal abstraction), since Analytica can leverage highly efficient array-operations during its computations. Thus, there is a trade-off to be had between memory consumption and speed. Also, if your model does utilize statistical functions in intermediate computations, you could still potentially loop over the Run index in a hybrid fashion. Similarly, if you model operates over an index, with some care a hybrid approach can still be utilized to loop over I (with the requirement that all variables compute a valid, although perhaps approximate, result for any subset of I).

The idea here is as follows. Suppose you have a very long index, I, that you wish to loop over (which might be Run in the large sample case). Rather than compute your model for each individual element of I, the hybrid approach replaces I with a subset of the original index at each iteration. So, if I := 1..1M, you might loop using I := 1..10000, then I := 10001..20000, and so on up to I := 990000..1000000. As a final step, you must concatenate the results back onto the original index.

The implementation of hybrid abstraction requires a reasonably high degree of Analytica expertise. The details are left up to the reader.