Aggregate(x, map: Array[i]; i, targetIndex: Index)

Optional parameters: Aggregate(x, map, i, targetIndex, type, positional, defaultValue)

Aggregate converts an array «x» indexed by a fine-grained index «i» (e.g., Days) into an array indexed by a coarser index «targetIndex» (e.g., Years). It aggregates (e.g. sums) all the elements of «x» that map to the same «targetIndex» value. The «map» parameter gives the value of «targetIndex» corresponding to each value of index «i» -- e.g. the Year containing each Day. Each cell of «map» must be an element of «targetIndex» -- or, if you specify the optional parameter positional: True, each cell of «map» specifies an integer position in «targetIndex».

By default, Aggregate sums all elements from «x» that map to corresponding value for «targetIndex». But, you can specify the optional Type parameter as Max, Min, Product, Average, Median, or any other array-reducing function -- i.e. a function reduces the number of dimensions by one. You can even provide a User-defined function that is an array-reducing function. You may specify the aggregation Type parameter as a text value -- e.g. "Product" -- or a handle to the function -- e.g. Handle(Product).

If «targetIndex» contains a value to which no element of «I» is mapped -- e.g. the index Days does not contain any Day in a particular value of Years -- the corresponding target cell in the result contains Null, or the «defaultValue» if you specify that parameter.

Library

Array functions

Examples

Index Months := Sequence(MakeDate(2009, 1, 1), MakeDate(2019, 12, 1), dateUnit:"M")
Index Years := 2009..2020
Year_by_months := DatePart(Months,"Y")
Revenue_by_month := { an array indexed by Months }
 
Revenue_by_year := Aggregate(Revenue_by_month, Year_by_months, Months, Years)

Aggregation types

The optional «type» parameter specifies the aggregation type. The parameter names an existing array-reducing function, either built-in or user-defined. You can supply either the textual name of the function, or a handle to the function (or even an array to these). Built-in functions that can be applied include Sum (default), Max, Min, Average or Mean, Product, Median, SDeviation, Variance, Skewness, Kurtosis, and Join.

Revenue_dev := Aggregate(Revenue_by_month,MonthToYear,Month,Year,Type:"SDeviation")
Peak_revenue:= Aggregate(Revenue_by_month,MonthToYear,Month,Year,Type:Handle(Max))
Ave_revenue := Aggregate(Revenue_by_month,MonthToYear,Month,Year,Type:"Average")

You can also create your own custom aggregation method in the form of a User-Defined Function. The function should accept an array parameter and an index, and should reduce the array so that the result eliminates the indicated index.

The following would all be examples of legal aggregation functions:

Function First(A:Array[I] ; I:Index) := A[@I=1]
Function Last(A:Array[I] ; I:Index) := A[@I=Size(I)]
Function Middle(A:Array[I] ; I:Index) := A[@I=Floor(Size(I)/2)+1]
Function PosMax(A:Array[I] ; I:Index) := ArgMax(A,I,position:true)

These would be used in the same fasion as other types:

Opening_Revenue := Aggregate(Revenue_by_month,MonthToYear,Month,Year,Type:"First")
Closing_Revenue := Aggregate(Revenue_by_month,MonthToYear,Month,Year,Type:"Last")
Best_Month := Aggregate(Revenue_by_month,MonthToYear,Month,Year,Type:Handle(PosMax))

If you desire to use "Count" as an aggregation type, you simply need to use a boolean expression for the first parameter, and do not need to specify the «type» parameter. For example, to count the number of months in each year with positive revenue:

Aggregate(Revenue_by_month>0,MonthToYear,Month,Year)

Note: Aggregation (aka conglomeration) functions can also be utilized by the MdTable function. The requirements are the same.

Positional Maps

The «map» parameters to specifies how each element in the original index «I» maps into the «targetIndex». This is usually a many-to-one mapping. Your «map» is necessarily indexed by «I», and each element of «map» specifies an element of «targetIndex». When specifying the target values, you can do so either by value or by position. The default assumes that map contains values from «targetIndex», but by specifying the optional «positional» parameter as true, «map» is interpreted as containing positions.

Variable M_to_Y_pos := Floor( (@Month-1)/12 ) + 1
Variable Annual_Rev := Aggregate(Revenue_by_month,M_to_Y_pos,Month,Year,positional:true)

The «map» may contain Null values. With a positional mapping, any «I»-position in which «map» is Null does not get included in the aggregated result. With a value-based mapping, these elements are included only if one of the elements of «targetIndex» is «null» (which is uncommon).

Default Value

It is possible that no data maps to a particular value/position in «targetIndex». This can occur when that value (or position) never occurs within «map», or when all data in «X» that would map to that position is Null-valued. (Note: Aggregate ignores null values in «X»).

For result cells with no data, «null» is returned. No warning is issued when there are result cells with no data. If you'd like to use a different value, such as 0, you can specify the value in the optional «defaultValue» parameter, e.g.:

Aggregate(X,map,I,J,type:"SDeviation",defaultValue:0)

Inverses of Aggregate (De-aggregation)

Since Aggregate is used to map data from a finer-grained index to a coarser-grained index, it is natural to ask how the inverse is accomplished -- namely, mapping from a coarse-grained index to a fine-grain index.

Just as with aggregate, there are many potential types of de-aggregation. For example, when de-aggregating annual data down to months, we might use the annual value for every month in that year, or split it uniformly among all the target months, or perhaps assign it to only one month, using zero for all other months in that year.

The following uses the annual value for every month in that year. In the example, Annual_val is indexed by Year, while the result is indexed by Month:

Annual_val[ Year = MonthToYear ]

To spread the value uniformly across all months, we can do this:

Var MonthsPerYear := Aggregate(1,MonthToYear,Month,Year);
(Annual_val / MonthsPerYear)[ Year = MonthToYear ]

More generally, if we have a weighting, w, indexed by Month, which specifies the proportion of annual revenue each month receives, we can use:

w * Annual_val[ Year = MonthsToYear ]

If you need to ensure that w is normalized for each month, use as the weight:

w := (w0 / Aggregate(1,MonthToYear,Month,Year)[Year=MonthToYear])

where w0 is the unnormalized weight. Using w0:=1 yields the earlier example of spreading evenly across all months for the year.

Using an array w that has a 1 in the first month of each year, and 0 for all remaining months, yields a de-aggregation that assigns the full annual value to the first month of each year:

Index Firsts := Unique(MonthToYear,Month);
(@[Firsts=Month]>0) * Annual_val[Year=MonthToYear]

Other methods for Aggregation

In many existing Analytica models, one will see the following method used for aggregation:

Sum( (MonthToYear=Year) * Revenue, Month )

This expression is functionally equivalent to:

Aggregate(Revenue,MonthToYear,Month,Year)

However, the use of Aggregate will evaluate much faster. When using the dot-product method with Sum, the intermediate expression (MonthToYear=Year) expands to a full 2-D array (Month x Year), which even though it is very sparse, still consumes lots of memory and time to process. In rough terms, the Sum-based method is O(n²) complexity, while Aggregate is only O(n).

The Frequency function can be used to aggregate, with either "sum" type aggregation (using the data itself as the weighting parameter) or "count" type aggregation. Frequency is in fact quite efficient for this purpose, but the usage for the purpose is unintuitive, making Aggregate the preferred alternative.

Slice Assignment can be used in a procedural-programming style to aggregate. This is a very general-purpose approach (can be employed for a variety of aggregation types) and can also be quite efficient. It also works in Analytica 4.1. The general approach is demonstrated here (for Max-type aggregation):

Var res:=Null;
For m:=Month Do (
   res[Year=MonthToYear] := Max( [res[Year=MonthToYear],Revenue[Month=m] ] )
);
res

Aggregate

Contents