Analytica Docs - User contributions [en]

INF, NAN, Null, and Undefined

2015-01-13T22:01:04Z

Psanford: Psanford moved page INF, NAN, Null, and Undefined to INF, NAN, and Null: Remove "Undefined" from the title

#REDIRECT [[INF, NAN, and Null]]

INF, NAN, and Null

2015-01-13T22:01:03Z

Psanford: Psanford moved page INF, NAN, Null, and Undefined to INF, NAN, and Null: Remove "Undefined" from the title

[[category:Concepts]]
[[Category:Doc Status C]] 

These are special values that Analytica returns in special conditions:

:'''Inf''' means infinity -- e.g., 1/0 -- or a real number larger than can be represented -- e.g. 1E307 * 100, because the maximum number it can handle is 1.796E308.
:'''-Inf''' rmeans negative infinity -- e.g., -1/0 -- or a number smaller than can be represented
:'''NAN''' means "Not A Number", where the result is numeric, but not a real number or infinity -- e.g., [[Sqrt]](-1) or 0/0
:'''Null''' means that there is no such value. [[Slice]], [[Subscript]], [[SubIndex]], or [[MdTable]] return Null, for example, when trying to Slice out the nth slice over an Index with less than n values.

=== Computations with special values ===

Analytica follows ANSI (Association of National Standards Institutes) standards for computing with these special values, where applicable.

:1/Inf → 0
:1/(-Inf) → 0
:Inf + Inf → Inf
:Inf - Inf → NAN

When NULL appears in scalar operations, it generally produces a warning and evaluates to NULL, for example:
:10 + NULL → NULL
:NULL - 10 → NULL
:1 AND NULL → NULL

When NULL appears in an array operation, the result should be ignored. The common array reduction functions in Analytica 4.1 tend to support this as of release 4.1, but not prior to that, and not all array function support this. These examples demonstrate (assume A is indexed by I as indicated).
:{|border="1"
! I: !! 1 !! 2 !! 3 !! 4 !! 5
|-
! A: || 8 || NULL || 4 || NULL || 0
|}

:[[Sum]](A,I) → 12
:[[Average]](A,I) → 4
:[[Join]](A,I,',') → "8,4,0"

In Analytica 4.1, these array functions ignore Null: [[Sum]], [[Min]], [[Max]], [[ArgMin]], [[ArgMax]], [[Product]], [[Average]], [[JoinText]], [[Irr]], [[Npv]]. Also [[Regression]] ignores data points which have Y=Null.

Array functions [[Sum]], [[Min]] and [[Max]] also accept an optional parameter ''IgnoreNaN'' to ignore NaN values (otherwise they tend to be propagated).

Special Handling of Date Values

2014-12-17T21:01:38Z

Psanford:

Analytica includes group of [[Date Functions]] specifically designed to create and manipulate date values. But there are also some common functions which, although they are typically applied to standard numerical values, include optional parameters to handle dates. This page links to all functions that handle dates.
= The dateUnit Parameter =

The ''dateUnit'' parameter is available in several rounding functions. It must be called explicitly and replaces the ''digits'' parameter.

Functions that use the ''dateUnit'' parameter:
* [[Floor|Floor()]]
* [[Ceil|Ceil()]]
* [[Round|Round()]]
* [[Sequence|Sequence()]]

dateUnit values include:
- 'Y' for Year
- 'Q' for Quarter
- 'M' for Month
- 'WD' for Weekday
- 'D' for Day
- 'h' for Hour
- 'm' for minute
- 's' for second

Note that Month and minute are distinguished by case.

;Examples:
Round up to the next year: Ceil(MakeDate(2014, 12,17), dateUnit: 'Y') → 1-Jan-2015

Round to the nearest top of the minute: Round(Now(withTime:1), dateUnit: 'm') → 17-Dec-2014 11:32:00 AM

List the next 5 Groundhog Days: Sequence(MakeDate(2015,2,2), MakeDate(2019,2,2), dateUnit: 'Y')
→ [2-Feb-2015, 2-Feb-2016, 2-Feb-2017, 2-Feb-2018, 2-Feb-2019]

= Creating Dates =
It is possible to create a date value without help from any function. A value will be recognized as a date if it follows the syntax dd-EEE-yyyy 

Where:

 dd is one or two digits for day of month

 EEE is a three character text code for the month in the default language set in the OS. (English in the example below)

 yyyy is four digits for the year

;Example: 04-DEC-2015

The bare date syntax must be used on its own. It cannot be embedded within an expression.

To explicity define a date you can use one of the following functions:
* [[ParseDate|ParseDate()]]
* [[MakeDate|MakeDate()]]
= Importing Dates =
[[DbQuery|DbQuery()]] and [[SpreadsheetRange|SpreadsheetRange()]] can import date values into Analytica. Both recognize date type values in the source data.

By default, DbQuery does not convert these to dates when importing but you can change this preference using the DatesAsText parameter. More details [[DbQuery#Date_and_Time_data|here]].

 SpreadsheetRange() always preserves date values when they are imported. There are no optional parameters to think about in this case.
= Dedicated Date Functions =
The section above summarized special handling of dates using multi-purpose functions. Dedicated date functions are documented on the [[Date Functions]] page.

Special Handling of Date Values

2014-12-17T19:44:54Z

Psanford:

Special Handling of Date Values

2014-12-17T19:23:30Z

Psanford:

Special Handling of Date Values

2014-12-13T07:13:34Z

Psanford:

Special Handling of Date Values

2014-12-13T07:08:13Z

Psanford:

Special Handling of Date Values

2014-12-13T06:40:14Z

Psanford:

Special Handling of Date Values

2014-12-13T06:35:25Z

Psanford: Created page with "Analytica includes group of Date Functions specifically designed to create and manipulate date values. But there are also some common functions which, although they are typica..."

Analytica includes group of Date Functions specifically designed to create and manipulate date values. But there are also some common functions which, although they are typically applied to standard numerical values, include optional parameters to handle dates. This page links to all functions that handle dates.

Date Functions

2014-12-13T06:24:34Z

Psanford:

[[Category:Date and Time Functions]]
[[Category:Doc Status C]] 
A date in Analytica is represented as the number of days since Jan 1, 1904 (or 1900), as in Excel. You can enter or display a date in almost any format -- using US or ROW (rest of the world) conventions. A time is expressed as a fraction of a day. You can set the display format for a number as a date in wthe Number format dialog. You can create a date or time with MakeDate() and MakeTime(), get the current date or time with Today(), select an element of a date with DatePart(0), and add days, weeks, months, or years to a date with DateAdd(). See below for details.

===Date formats===

You can control whether and how a number is displayed as a date in the '''Number Format''' dialog, available from the '''Result''' menu:

[[Image:Number format.png|400px]]

The '''Date''' format in the '''Number Format''' dialog offers these options:

:'''Short''' -- e.g., 8/5/2006
:'''Abbrev''' -- e.g., Aug-5-2006
:'''Long''' -- e.g., Thursday, 05 August, 2006
:'''Full''' -- e.g., "5-Aug-2006 3:50:21 PM" ''{ introduced in release 4.2.3 }''
The actual formats for the above dates depend on settings in the '''Regional and Language Options''' from the Windows '''Control Panel'''. See below for details.
:'''Custom''' -- lets you select an existing custom format or make a new one. These are some examples
::dd-MM-yy → 05-8-06
::'Q'Q YYYY → Q2 2006
::www, d MMM yyyy → Thu, 5 Aug 2006
::wwww, d of MMMM, yyyy → Thursday, 5 of August, 2006
::d-MMM-yyyy hh:mm:ss tt → 5 Aug 2006 05:45:00 PM ''{ "hh:mm:ss tt" requires 4.1 }''

The Custom date format uses these letter codes (standard conventions from Microsoft Windows):
::d: numerical day of the month -- 1, 2,... 31
::dd: numeric day of the month with two digits -- 01, 02,...31
::ddd: abbreviated ordinal day of month - 1st, 2nd, ..., 31st
::dddd: ordinal day of month -- first, second, ..., thirty-first
::Dddd: capitalized ordinal day of month -- First, Second, ... Thirty-first
::www: weekday in three letters -- Mon, Tue,.. Sun
::wwww: weekday in full -- as Monday, Tuesday, ... Sunday
::M: month as a number -- 1, 2, ... 12
::MM: month as two-digit number -- 01, 02, ...12
::MMM: month as three letter name -- Jan, Feb, ... Dec
::MMMM: month as full name -- January, February, ... December
::q: quarter as one digit -- 1, 2, 3, 4
::yy: year as two digits -- e.g., 99, 00, 01
::yyyy: year as four digits -- e.g., 1999, 2000, 2001
::Any other characters, including space, appear literally as given.
::To show a character literally that is also a code, enclose it in quotes, e.g. 'q'q → q2

If you specify any date format for an input variable or Edit table, you can enter dates in any acceptable date format -- no matter which date format is specified. For example, a variable with a date format, interprets "9/11/2001" as "11 September, 2001" on a computer set to USA region or "9 November, 2001" elsewhere. With no date format, it interprets "9/11/2001"as ((9 divided by 11) divided by 2001) = 0.000409!

=== Regional and Language settings===

The names of days and months and the formats used for Short, Abbrev and Long dates depend on the regional settings for Windows. So, a user in the US may see a short date as 9/11/2001, while a user in Denmark may see 11/9/2001. You can review the settings in '''Regional and Language Options''' from the Windows '''Control Panel'''. To modify them, click '''Customize''' button and select teh '''Date''' tab or '''Languages''' tab. For example, if you set the language to ''Spanish (Argentina)'', Makedate(2007, 2, 3) with the '''Long''' date setting displays as:
:Sábado, 03 de Febrero de 2007.

===Date values and the date origin===

Analytica represents a date as a ''date value'' -- that is, the number of days since the ''date origin''. By default the date origin is Jan 1, 1904 -- as used by most Macintosh applications, including Excel on Macintosh, and all releases of Analytica on Macintosh and Windows up to Analytica 3.1. If you check '''Use Excel date origin''' in the '''Preferences...''' dialog, the date origin is Jan 1, ''1900'' -- as used in Excel on Windows (unless reset) and most other Windows software.

With '''Use Excel date origin'' checked, the numeric value of dates are the same in Analytica and Excel for Windows for dates falling on or after 1 Mar 1900. Because of a bug in Excel, in which Excel incorrectly treats Feb 29, 1900 is a valid day (1900 was not really a leap year), dates falling before that date will not have the same numeric index in Analytica as they do in Excel.

The integer portion of the date value represents the number of days since the date origin. The fractional portion, if present, represents the time as a fraction of a 24-hour day starting at midnight. The time portion is utilized in release 4.1. Excel utilizes the time fraction in the same manner.

When using models containing dates or date functions from Analytica releases 3.1 or earlier, you should keep '''Use Excel date origin''' unchecked. If you want to paste or link values from Excel for Windows or other Windows software to or from Analytica, you should check this option.

Analytica can handle dates from 1 AD to well beyond 9999 AD. Dates earlier than the date origin are represented as negative integers. Dates use the Gregorian calendar: Years divisible by 4 are leap years, except those divisible by 100 which are not leap years, except those divisible by 400 which are leap years.

You can simply add an integer ''n'' to a date to get the date ''n'' days ahead.

===MakeDate(year, month, day)===

Gives the date value for the date with that '''year''', '''month''', and '''day'''. If omitted, '''month''' and '''day''' default to 1.

;Examples:
MakeDate(2007,5,15) → 15-May-2007
MakeDate(2000) → 1-Jan-2000

;Requirements: year, month, and date should be positive integers, or coercible to positive numbers.

;[[Syntax]]: MakeDate(year: Coerce Positive; month, day: Optional Coerce Positive)

;Library: [[Special functions]]

===MakeTime(h,m,s)===

Gives the time of day as an elapsed fraction of the day, i.e., a number between 0 and 0.99999. The allowed range on the parameters are: 0<=h<=23, 0<=m<=59 and 0<=s<=59, with the exception that h can exceed 23 when encoding a duration of more than a day, in which case the result will be greater than 1.

;Examples
MakeTime(15,30,00) → 0.645833333 { 03:30:00 PM }

===DatePart(date, part)===

Gives the year, month, or day as a number, given a date value '''date'''. When '''part'''='Y', it gives the four digit year as a number, such as 2006. When '''part'''='M', it gives a number between 1 and 12. When '''part'''='D', it gives a number between 1 and 31. When '''part'''='w', it returns a number from between 1 (Sunday) and 7 (Saturday).

;Example:
DatePart(MakeDate(2006, 2, 28), 'D') -> 28

;Expects: '''date''' as a date number, and '''part''' as 'Y', 'M', or 'D' (upper- or lowercase, by 'M' must be upper).

;[[Syntax]]: DatePart(date: Numeric; part: Text)

;Library: [[Special functions]]

;Less Common Options: '''part''' may also be: 'MMM' (returns e.g., 'Jan'), 'MMMM' (e.g., 'January'), 'ddd' (e.g., '1st'), 'dddd' (e.g., 'first'), 'Dddd' (e.g., 'First'), 'w' (e.g., 1 for Sunday, 2 for Monday, ..., 7 for Saturday), 'www' (e.g., 'Mon'), 'wwww' (e.g,. 'Monday'), 'q' (1 to 4, e.g., 1 for Jan-Mar).

;Time components (requires 4.1): 'H' (hour 0 to 23), 'h' (hour 1 to 12), 'm' (minute 0 to 59}, 's' (second 0 to 59), 'HH', 'hh', 'mm', 'ss' (two digit text, e.g., "03"), ' H', ' h', ' m', ' s', 't', 'tt' ("AM" or "PM").

;Elapsed days (requires 4.1): 'wd', 'wd+', 'wd-' (number of weekdays since the date origin. 'wd' and 'wd+' counts the indicated day, 'wd-' does not count the indicated day), '#d' (day number in current year), '#w' (week number in current year, week starts on Sunday), '#wm' (week number in current year, week starts on Monday), 'e#w' and 'e#wm' (European convention, week number in current year, week 1 is the first week with at least 3 days, with 'e#w' week starts on Sunday, with 'e#wm' week starts on Monday).

;More Examples and Tips:

This makes a sequence of weekdays landing between date1 and date2:
Index J:=date1..date2 do Subset( DatePart(J,"w")>=2 and DatePart(J,"w")<=6 )

This computes the number of weekdays between day1 and day2, including day1 and day2 (if day1 or day2 land are weekdays). When day2<day1 the result is negative. The example requires 4.1:
DatePart(day2,"wd+") - DatePart(day1,"wd-")

===DateAdd(date, offset, unit)===

Gives a date value that is '''offset''' from '''date''' by '''unit''' "Y" (years), "Q" (quarters) "M" (months) "D" (days), "WD" (weekdays, skipping over Saturdays and Sundays. It also works for time increments: "h" (hours), "m" (minutes) or "s" (seconds).

;Examples:
DateAdd is especially useful for generating a sequence of dates for a time index:

DateAdd(MakeDate(2006, 1, 1), 0..12, "M")
-> [1-Jan-2006, 1-Feb-2006, 1-Mar-2006, ... 1-Jan-2007]

Note: the actual display format for the dates depends on what was set in Number format.

If an offset would appear to go past the end of a month, such as
DateAdd(MakeDate(2004, 2, 29), 1, 'Y') -> 2005-Feb-28
DateAdd(MakeDate(2006, 10, 31), 1, 'M') -> 2006-Nov-30
it returns the last day of the month. In the first example, the date 2005-Feb-29 does not exist, and in the second example the date 2006-Nov-31 doesn't exist, so in each case, it returns the last day of the month.

Adding a day offset, DateAdd(date, n, "D"), is equivalent to date+n, since date is represented as an integer. DateAdd(date, n, "WD") adds the specified number of weekdays to the first weekday equal to or falling after '''date'''.

Use a negative ''offset'' to subtract units from a date.

;Expects: '''date''' and '''offset''' to be numbers, and '''unit''' to be 'Y', 'Q', 'M', 'D', 'WD', 'h', 'm', or 's'. The unit is not case sensitive except for 'M' (months) and 'm' (minutes).

;[[Syntax:]] DateAdd(date, offset: Number; unit: Text)

;Library: [[Special functions]]

; More examples and tips:

MakeDate(2007, 2, 10) --> Saturday, 2007-Feb-10
DateAdd(MakeDate(2007, 2, 10), 0, "WD") --> Monday, 2007-Feb-12
DateAdd(MakeDate(2007, 2, 10), -1, "WD") --> Friday, 2007-Feb-9
DateAdd(MakeDate(2007, 2, 10),365, "D") --> Sunday, 2008-Feb-10
MakeDate(2007, 2, 10) + 365 --> Sunday, 2008-Feb-10
DateAdd(MakeDate(2007, 2, 10), 365, "WD") --> Monday, 2008-July-7

This generates a series of dates with the 1st and 15th of each month:
var J := 0..24;
DateAdd(MakeDate(2006, 1, IF Mod(J, 2)=0 Then 1 Else 15), Floor(J/2), "M")
-->
[2006-Jan-1, 2006-Jan-15, 2006-Feb-1, 2006-Feb-15, 2006-Mar-1, ..., 2007-Jan-1]

===Today()===

Returns the date number (number of days since base date, usually Jan 1, 1904, as in Excel) for the day on which the function is evaluated. Unlike other functions, it gives a different value every day the model is run.

;Time(withTime, utc)

Today() accepts two optional flags: Setting ''WithTime'' to true returns the current time of day in the fractional part of the result, by default for your local time zone. If you set ''utc'' to true, it returns the date and time in Universal Time Coordinated (UTC), a.k.a. Greenwich Mean Time.

Note that the result is the date and time when the function is evaluated in this session. The result is cached in the value of a variable that uses Today(). It will not be updated when the date or time changes unless you trigger a re-evaluation -- e.g. by reloading the model or by changing another input to the variable that uses Today().

;Library: [[Special functions]]

;Library: [[Special functions]]

See also: [[Special Handling of Date Values]]

Category:System Variables

2014-11-05T00:01:24Z

Psanford:

==System Variables==

System variables are variables predefined in Analytica. They help control all kinds of behavior and appearance of Analytica. Most of the options in the [[Preferences]] dialog and other dialogs set System variables. For example, in the '''Uncertainty Sample''' tab in '''Uncertainty Setup''' dialog, you can set the Sample size. This sets a System variable Samplesize.

You can refer to System variables in expressions. For example, you could define a function to estimate the standard error in the estimate of the mean of x of a Monte Carlos sample as:
Function Error_in_mean(x) := Sdeviation(x)/Sqrt(SampleSize)

You can also change the values of some System variables directly by assigning to them in the [[Typescript]] or in a [[Script]] attribute of Button or user input variable.

Some System variables are not accessible directly from user-interface dialogs (unlike Samplesize, for example). Many of these are esoteric or obsolete, but a few listed below are often quite useful. The best way to modify these is via the [[Typescript]] window. Press Ctrl-' (control key and single quote) to open this Window.

See also:
* [[System Variables|System Variables]] described in the Scripting Guide.

There are more than 160 system variables. The full list can be viewed by typing list sysvar in the typescript window. Some are esoteric or obsolete and only exist for compatibility with legacy models. The most useful ones are those that are not already accessible through the UI and are potentially useful for access in the typescript window either for setting new values or for referencing the current value. This page focuses on the most useful system variables.

*AttributeFontSize
This sets the font size in the attribute pane and object windows. This setting is also accessible in the UI by selecting Preferences from the Edit menu. The check box labeled Large text in attributes & tables toggles between 10 point and 13 point. (The UI setting also affects TableFontSize.) Other font sizes can be chosen through the typescript window. Since this is a preference value, its value sticks to Analytica rather than being saved to a particular model.

*TableFontSize
This sets the font size in tables and results. This setting is also accessible in the UI by selecting Preferences from the Edit menu. The check box labeled Large text in attributes & tables toggles between 10 point and 13 point. (The UI setting also affects AttributeFontSize.) Other font sizes can be chosen through the typescript window. Since this is a preference value, its value sticks to Analytica rather than being saved to a particular model.

*AutoSaveFilename
This is a non-editable read-only variable. It shows the file path Analytica is currently using to save the recovery file. Referencing this variable in the typescript window is a convenient way to locate the last saved recovery file.

*InternalSDKErrTries
The Frontline optimization engines sometimes fail to compute, and return an SDK error. This is a known issue that Frontline has ameliorated but not fully resolved (as of Analytica 4.5). The occurrence of these errors is not predictable. When one occurs, it is possible that the next optimization with run with no problems. This system variable determines the number of consecutive errors and automatic restarts Analytica will allow without faulting. If you are getting frequent SDK errors with InternalSDKErrTries = 0, try setting it to 5 or so. The value of this variable sticks to an individual model. The default value for new models is 0.

*It
When you are using typescript for calculations by entering Analytica expressions, It refers to the last result. For example:
> pi*2 returns 6.28
> It * 2 then returns 12.57

*Sv__DateOriginOffset
This setting is accessible in the UI by selecting Preferences from the Edit menu. If Use Excel date origin is unchecked, it will use Jan 1, 1904 as used on Macintosh, corresponding to an offset of zero. If Use Excel date origin is checked, it will use Jan 1, 1900 as used on Excel on Windows. This corresponds to a date offset of -1462.

*Sys_AllNullTreatment
This is useful for legacy models made with Analytica versions before 4.4. 4.4 changed the way Analytica handles null values is some functions. See [[What's new in Analytica 4.4?]].

*Sys_AnimateWindows
When set to 1, Analytica will animate windows as they open and close. This is useful for presentations since it gives a visual cue whenever a module is opened to a window. This feature is deactivated by default (Sys_AnimateWindows = 0) because some graphics processors fail to redraw properly when it is activated. If your system does not exhibit this problem, you may find it preferable.

*Sys_DbDatesAsText
This affects how the DBQuery() function handles date values. If dates are imported numerically, they will be processed as dates in Analytica functions. But if this creates problems, set Sys_DbDatesAsText = 1 to suppress date intelligence and handle the imported values as text.

*Sys_ShowFontsOnPopup
This is active (1) by default. When you select Set Node Style... from the diagram menu, then select Font, you will see the font names displayed in the actual font styles. If this feature is taking up too much processing time or memory, set this variable to 0.

*Sys_TableCellDefault
This variable sets the default value for table cells when no value has been set the Cell Default attribute. In other words, Sys_TableCellDefault is the default to the default. For new models, the value of Sys_TableCellDefault is 0. If you change it, the new value will be saved with the model. To designate empty text for the default, do not type anything after the colon in the typescript command. ( Sys_TableCellDefault: )
Regardless of Sys_TableCellDefault , you can always use the Cell Default attribute to establish a default value for an individual table. First activate the Cell Default attribute by selecting Attributes... from the Object window. Place a check box next to Cell Default. The attribute will now appear in the object window for tables.

*Sys_WarnUndefCell
This Boolean variable enables a warning when you try to evaluate a table with one or more undefined cells. For new models, this value is set to 1 (warning enabled). If you edit this variable, the new value will be saved with the model.

*WarnOnNonIndexedOp
This Boolean variable enables a warning when you omit the index attribute from array reducing functions. Sum() is an example of an array reducing function. It normally has two attributes: The array and the index to operate over. For example: Sum(cost_table, item_index). Analytica will always warn when the array has two or more dimensions and you omit the second parameter. But there is no ambiguity you are operating on a one-dimensional array or an index. In this case, Analytica will either warn or not depending on the value of WarnOnNotIndexedOp . It is always a good idea to include the index parameter regardless of the dimensions of the array. This way you can add new dimensions to the existing model without introducing ambiguities. Although WarnOnNonIndexedOp is not enabled for new models, it is a good idea to enable it if you want to enforce this guideline.

Category:System Variables

2014-10-20T19:37:10Z

Psanford:

==System Variables==

System variables are variables predefined in Analytica. They help control all kinds of behavior and appearance of Analytica. Most of the options in the [[Preferences]] dialog and other dialogs set System variables. For example, in the '''Uncertainty Sample''' tab in '''Uncertainty Setup''' dialog, you can set the Sample size. This sets a System variable Samplesize.

You can refer to System variables in expressions. For example, you could define a function to estimate the standard error in the estimate of the mean of x of a Monte Carlos sample as:
Function Error_in_mean(x) := Sdeviation(x)/Sqrt(SampleSize)

You can also change the values of some System variables directly by assigning to them in the [[Typescript]] or in a [[Script]] attribute of Button or user input variable.

Some System variables are not accessible directly from user-interface dialogs (unlike Samplesize, for example). Many of these are esoteric or obsolete, but a few listed below are often quite useful. The best way to modify these is via the [[Typescript]] window. Press Ctrl-' (control key and single quote) to open this Window.

See also:
* [[System Variables|System Variables]] described in the Scripting Guide.

There are more than 160 system variables. The full list can be viewed by typing list sysvar in the typescript window. Some are esoteric or obsolete and only exist for compatibility with legacy models. The most useful ones are those that are not already accessible through the UI and are potentially useful for access in the typescript window either for setting new values or for referencing the current value. This page focuses on the most useful system variables.

*AttributeFontSize
This sets the font size in the attribute pane and object windows. This setting is also accessible in the UI by selecting Preferences from the Edit menu. The check box labeled Large text in attributes & tables toggles between 10 point and 13 point. Other font sizes can be chosen through the typescript window. Since this is a preference value, its value sticks to Analytica rather than being saved to a particular model.

*TableFontSize
The font size (in points) for text shown in table results and edit tables.

*AutoSaveFilename
This is a non-editable read-only variable. It shows the file path Analytica is currently using to save the recovery file. Referencing this variable in the typescript window is a convenient way to locate the last saved recovery file.

*InternalSDKErrTries
The Frontline optimization engines sometimes fail to compute, and return an SDK error. This is a known issue that Frontline has ameliorated but not fully resolved (as of Analytica 4.5). The occurrence of these errors is not predictable. When one occurs, it is possible that the next optimization with run with no problems. This system variable determines the number of consecutive errors and automatic restarts Analytica will allow without faulting. If you are getting frequent SDK errors with InternalSDKErrTries = 0, try setting it to 5 or so. The value of this variable sticks to an individual model. The default value for new models is 0.

*It
When you are using typescript for calculations by entering Analytica expressions, It refers to the last result. For example:
> pi*2 returns 6.28
> It * 2 then returns 12.57

*Sv__DateOriginOffset
This setting is accessible in the UI by selecting Preferences from the Edit menu. If Use Excel date origin is unchecked, it will use Jan 1, 1904 as used on Macintosh, corresponding to an offset of zero. If Use Excel date origin is checked, it will use Jan 1, 1900 as used on Excel on Windows. This corresponds to a date offset of -1462.

*Sys_AllNullTreatment
This is useful for legacy models made with Analytica versions before 4.4. 4.4 changed the way Analytica handles null values is some functions. See [[What's new in Analytica 4.4?]].

*Sys_AnimateWindows
When set to 1, Analytica will animate windows as they open and close. This is useful for presentations since it gives a visual cue whenever a module is opened to a window. This feature is deactivated by default (Sys_AnimateWindows = 0) because some graphics processors fail to redraw properly when it is activated. If your system does not exhibit this problem, you may find it preferable.

*Sys_DbDatesAsText
This affects how the DBQuery() function handles date values. If dates are imported numerically, they will be processed as dates in Analytica functions. But if this creates problems, set Sys_DbDatesAsText = 1 to suppress date intelligence and handle the imported values as text.

*Sys_ShowFontsOnPopup
This is active (1) by default. When you select Set Node Style... from the diagram menu, then select Font, you will see the font names displayed in the actual font styles. If this feature is taking up too much processing time or memory, set this variable to 0.

*Sys_TableCellDefault
This variable sets the default value for table cells when no value has been set the Cell Default attribute. In other words, Sys_TableCellDefault is the default to the default. For new models, the value of Sys_TableCellDefault is 0. If you change it, the new value will be saved with the model. To designate empty text for the default, do not type anything after the colon in the typescript command. ( Sys_TableCellDefault: )
Regardless of Sys_TableCellDefault , you can always use the Cell Default attribute to establish a default value for an individual table. First activate the Cell Default attribute by selecting Attributes... from the Object window. Place a check box next to Cell Default. The attribute will now appear in the object window for tables.

*Sys_WarnUndefCell
This Boolean variable enables a warning when you try to evaluate a table with one or more undefined cells. For new models, this value is set to 1 (warning enabled). If you edit this variable, the new value will be saved with the model.

*WarnOnNonIndexedOp
This Boolean variable enables a warning when you omit the index attribute from array reducing functions. Sum() is an example of an array reducing function. It normally has two attributes: The array and the index to operate over. For example: Sum(cost_table, item_index). Analytica will always warn when the array has two or more dimensions and you omit the second parameter. But there is no ambiguity you are operating on a one-dimensional array or an index. In this case, Analytica will either warn or not depending on the value of WarnOnNotIndexedOp . It is always a good idea to include the index parameter regardless of the dimensions of the array. This way you can add new dimensions to the existing model without introducing ambiguities. Although WarnOnNonIndexedOp is not enabled for new models, it is a good idea to enable it if you want to enforce this guideline.

Category:System Variables

2014-10-20T19:34:58Z

Psanford:

==System Variables==

System variables are variables predefined in Analytica. They help control all kinds of behavior and appearance of Analytica. Most of the options in the [[Preferences]] dialog and other dialogs set System variables. For example, in the '''Uncertainty Sample''' tab in '''Uncertainty Setup''' dialog, you can set the Sample size. This sets a System variable Samplesize.

You can refer to System variables in expressions. For example, you could define a function to estimate the standard error in the estimate of the mean of x of a Monte Carlos sample as:
Function Error_in_mean(x) := Sdeviation(x)/Sqrt(SampleSize)

You can also change the values of some System variables directly by assigning to them in the [[Typescript]] or in a [[Script]] attribute of Button or user input variable.

Some System variables are not accessible directly from user-interface dialogs (unlike Samplesize, for example). Many of these are esoteric or obsolete, but a few listed below are often quite useful. The best way to modify these is via the [[Typescript]] window. Press Ctrl-' (control key and single quote) to open this Window.

See also:
* [[System Variables|System Variables]] described in the Scripting Guide.

There are more than 160 system variables. The full list can be viewed by typing list sysvar in the typescript window. Some are esoteric or obsolete and only exist for compatibility with legacy models. The most useful ones are those that are not already accessible through the UI and are potentially useful for access in the typescript window either for setting new values or for referencing the current value. This page focuses on the most useful system variables.

*AttributeFontSize
This sets the font size in the attribute pane and object windows. This setting is also accessible in the UI by selecting Preferences from the Edit menu. The check box labeled Large text in attributes & tables toggles between 10 point and 13 point. Other font sizes can be chosen through the typescript window. Since this is a preference value, its value sticks to Analytica rather than being saved to a particular model.

*TableFontSize: The font size (in points) for text shown in table results and edit tables.

*AutoSaveFilename
This is a non-editable read-only variable. It shows the file path Analytica is currently using to save the recovery file. Referencing this variable in the typescript window is a convenient way to locate the last saved recovery file.

*InternalSDKErrTries
The Frontline optimization engines sometimes fail to compute, and return an SDK error. This is a known issue that Frontline has ameliorated but not fully resolved (as of Analytica 4.5). The occurrence of these errors is not predictable. When one occurs, it is possible that the next optimization with run with no problems. This system variable determines the number of consecutive errors and automatic restarts Analytica will allow without faulting. If you are getting frequent SDK errors with InternalSDKErrTries = 0, try setting it to 5 or so. The value of this variable sticks to an individual model. The default value for new models is 0.

*It
When you are using typescript for calculations by entering Analytica expressions, It refers to the last result. For example:
> pi*2 returns 6.28
> It * 2 then returns 12.57

*Sv__DateOriginOffset
This setting is accessible in the UI by selecting Preferences from the Edit menu. If Use Excel date origin is unchecked, it will use Jan 1, 1904 as used on Macintosh, corresponding to an offset of zero. If Use Excel date origin is checked, it will use Jan 1, 1900 as used on Excel on Windows. This corresponds to a date offset of -1462.

*Sys_AllNullTreatment
This is useful for legacy models made with Analytica versions before 4.4. 4.4 changed the way Analytica handles null values is some functions. See [[What's new in Analytica 4.4?]].

*Sys_AnimateWindows
When set to 1, Analytica will animate windows as they open and close. This is useful for presentations since it gives a visual cue whenever a module is opened to a window. This feature is deactivated by default (Sys_AnimateWindows = 0) because some graphics processors fail to redraw properly when it is activated. If your system does not exhibit this problem, you may find it preferable.

*Sys_DbDatesAsText
This affects how the DBQuery() function handles date values. If dates are imported numerically, they will be processed as dates in Analytica functions. But if this creates problems, set Sys_DbDatesAsText = 1 to suppress date intelligence and handle the imported values as text.

*Sys_ShowFontsOnPopup
This is active (1) by default. When you select Set Node Style... from the diagram menu, then select Font, you will see the font names displayed in the actual font styles. If this feature is taking up too much processing time or memory, set this variable to 0.

*Sys_TableCellDefault
This variable sets the default value for table cells when no value has been set the Cell Default attribute. In other words, Sys_TableCellDefault is the default to the default. For new models, the value of Sys_TableCellDefault is 0. If you change it, the new value will be saved with the model. To designate empty text for the default, do not type anything after the colon in the typescript command. ( Sys_TableCellDefault: )
Regardless of Sys_TableCellDefault , you can always use the Cell Default attribute to establish a default value for an individual table. First activate the Cell Default attribute by selecting Attributes... from the Object window. Place a check box next to Cell Default. The attribute will now appear in the object window for tables.

*Sys_WarnUndefCell
This Boolean variable enables a warning when you try to evaluate a table with one or more undefined cells. For new models, this value is set to 1 (warning enabled). If you edit this variable, the new value will be saved with the model.

*WarnOnNonIndexedOp
This Boolean variable enables a warning when you omit the index attribute from array reducing functions. Sum() is an example of an array reducing function. It normally has two attributes: The array and the index to operate over. For example: Sum(cost_table, item_index). Analytica will always warn when the array has two or more dimensions and you omit the second parameter. But there is no ambiguity you are operating on a one-dimensional array or an index. In this case, Analytica will either warn or not depending on the value of WarnOnNotIndexedOp . It is always a good idea to include the index parameter regardless of the dimensions of the array. This way you can add new dimensions to the existing model without introducing ambiguities. Although WarnOnNonIndexedOp is not enabled for new models, it is a good idea to enable it if you want to enforce this guideline.

Category:System Variables

2014-10-17T08:21:46Z

Psanford:

Analytica System Variables.

See also:
* [[System Variables|System Variables]] described in the Scripting Guide.

There are more than 160 system variables. The full list can be viewed by typing list sysvar in the typescript window. Some are esoteric or obsolete and only exist for compatibility with legacy models. The most useful ones are those that are not already accessible through the UI and are potentially useful for access in the typescript window either for setting new values or for referencing the current value. This page focuses on the most useful system variables.

*Attribute Font Size
This sets the font size in the attribute pane and object windows. This setting is also accessible in the UI by selecting Preferences from the Edit menu. The check box labeled Large text in attributes & tables toggles between 10 point and 13 point. Other font sizes can be chosen through the typescript window. Since this is a preference value, its value sticks to Analytica rather than being saved to a particular model.

*AutoSaveFilename
This is a non-editable read-only variable. It shows the file path Analytica is currently using to save the recovery file. Referencing this variable in the typescript window is a convenient way to locate the last saved recovery file.

*InternalSDKErrTries
The Frontline optimization engines sometimes fail to compute, and return an SDK error. This is a known issue that Frontline has ameliorated but not fully resolved (as of Analytica 4.5). The occurrence of these errors is not predictable. When one occurs, it is possible that the next optimization with run with no problems. This system variable determines the number of consecutive errors and automatic restarts Analytica will allow without faulting. If you are getting frequent SDK errors with InternalSDKErrTries = 0, try setting it to 5 or so. The value of this variable sticks to an individual model. The default value for new models is 0.

*It
When you are using typescript for calculations by entering Analytica expressions, It refers to the last result. For example:
> pi*2 returns 6.28
> It * 2 then returns 12.57

*Sv__DateOriginOffset
This setting is accessible in the UI by selecting Preferences from the Edit menu. If Use Excel date origin is unchecked, it will use Jan 1, 1904 as used on Macintosh, corresponding to an offset of zero. If Use Excel date origin is checked, it will use Jan 1, 1900 as used on Excel on Windows. This corresponds to a date offset of -1462.

*Sys_AllNullTreatment
This is useful for legacy models made with Analytica versions before 4.4. 4.4 changed the way Analytica handles null values is some functions. See [[What's new in Analytica 4.4?]].

*Sys_AnimateWindows
When set to 1, Analytica will animate windows as they open and close. This is useful for presentations since it gives a visual cue whenever a module is opened to a window. This feature is deactivated by default (Sys_AnimateWindows = 0) because some graphics processors fail to redraw properly when it is activated. If your system does not exhibit this problem, you may find it preferable.

*Sys_DbDatesAsText
This affects how the DBQuery() function handles date values. If dates are imported numerically, they will be processed as dates in Analytica functions. But if this creates problems, set Sys_DbDatesAsText = 1 to suppress date intelligence and handle the imported values as text.

*Sys_ShowFontsOnPopup
This is active (1) by default. When you select Set Node Style... from the diagram menu, then select Font, you will see the font names displayed in the actual font styles. If this feature is taking up too much processing time or memory, set this variable to 0.

*Sys_TableCellDefault
*Sys_WarnUndefCell
*TableFontSize
*TableType
*Verbosity
*WarnOnNonIndexedOp

Additional libraries

2012-09-06T17:25:11Z

Psanford: /* Data Standardization Library */

(Up to [[Analytica Wiki]])

= Individual functions with examples =

* [[media:Recumulate example.ana|Recumulate example.ana]] : Like [[Cumulate]], but resets to zero at selected points.

= Libraries Not Included with the Analytica Installer =

== [[Large Sample Library: User Guide|Large Sample Library]] ==

[[image:Large sample library.png]]

The Large Sample Library is an Analytica library that lets you run a Monte Carlo simulation for large models or a large sample size that might otherwise exhaust computer memory, including virtual memory. It breaks up a large sample into a series of batch samples, each small enough to run in memory. For selected variables, known as the Large Sample Variables or LSVs, it accumulates the batches into a large sample. You can then view the probability distributions for each LSV using the standard methods — confidence bands, PDF, CDF, etc. — with the full precision of the large sample.

See [[Large Sample Library: User Guide]].

== [[The Sensitivity Analysis Library]] ==

[[The Sensitivity Analysis Library]] provides functions for analyzing the sensitivity of an output to each cell of each array-valued chance input, and locating those individual scalar inputs that have the greatest impact on the result. See [[The Sensitivity Analysis Library]] for documentation on using this library.

The library itself can be downloaded from [[media:Sensitivity Analysis Library.ana|Sensitivity Analysis Library.ana]], and an example model to demonstrate its usage is at [[media:Sensitivity Functions Examples.ana|Sensitivity Functions Examples.ana]].

== Model Documentation Library ==

'''Download: ''' [[media:Model Documentation Library.ana|Model Documentation Library.ana]]

'''Requires:''' Analytica 4.1.0 or later

Some people like to create a concise report on paper that contains all objects in their model along with descriptions or other selected attributes. The "Print Report" feature in Analytica can be used for this purpose, but is not at all concise and ends up placing each object window on a separate page. The ''Model Documentation Library'' allows you to select a module from your model and produce a result table containing every object within that module with its selected attributes such as title, description, units, definition, or user-defined attributes. This table can then be exported to Excel where you can format it nicely and print it. Thus, you can end up with a very concise report on paper.

To use this library, load your model and then select '''File-->Add Module...'''. Add the [[media:Model Documentation Library.ana|Model Documentation Library.ana]] module file using Embed. In that module, using the pulldown, select the top-level module for the report. Follow the instructions shown on the diagram. If you make changes to your model later, press the ''Update Model Documentation'' button to adjust the pulldown content.

== Units Conversion Library ==

'''Download: ''' [[media:Units conversion library.ana|Units conversion library.ana]]

The functions in this library provide conversions between different units -- for example, from feet to kilometers, or from Btu (British thermal units) to gigajoules of energy. Using this library relieves you from having to look up conversion factors. It also makes the model more transparent by making it clear where you are converting from one units to another -- instead of just embedding conversion constants in the formulas.

Use Conversion_Factor(oldUnits, newUnits) to give a conversion factor between two units.
If you omit newUnits, it will assume the standard units for that type (dimensions).
When specifying units , you must use a symbol (abbreviation) from the Units table.

If the units are of different dimensions, e.g. energy and power, or if it doesn't recognize the units, it will give an error.

Use Nom_to_real_dollars(yr, baseYr) to convert from nominal dollars for year yr to "real dollars" for specified baseYr. If you omit baseYr, it defaults to the Standard base year, initially 2000.

These conversion factors were provided on October 20, 2007 by Chris Marnay of the Lawrence Berkeley Laboratory, as part of his work on the SEDS project.

== Greatest Common Divisor functions ==

'''Download: ''' [[media:GCD function library.ana|GCD function library.ana]]

This library contains two [[User-Defined Functions]] for computing the greatest common divisor.

== DB Conversion Library ==

'''Download: ''' [[media:Db conversion lib.ana|DB conversion lib.ana]]

Press a button in this library to transform all the variables defined using [[DbQuery]], [[DbLabels]] and [[DbTable]] to literal data. Variables and indexes defined using [[DbQuery]] or [[DbLabels]] are transformed to list definitions, and those defined using [[DbTable]] are transformed into edit tables. The transformation has the effect of breaking the connection to the external database.

The library is limited in its scope. It only works when all calls to [[DbQuery]], [[DbLabels]] and [[DbTable]] occur at the top level of variable nodes. Do not attempt to use it if you have calls to these functions embedded within larger expressions or within [[User-Defined Functions]].

The purpose of this library is to make it possible to send a copy of your model to another person (including to Lumina Tech Support) who does not have access to your external database, or who does not have Analytica Enterprise.

'''''Use with extreme caution!!''''': Make a copy of your original model ''before'' adding and executing this module. After running the transformation, be sure to use '''File→Save As...''' to save the transformed model under a different filename, so you don't clobber your original model.

== Data Standardization Library ==

'''Download: ''' [[media:Data Standardization Library.ana|Data Standardization Library.ana]]

Imported data is often inconsistent. This library allows you to choose what the "standard" values should be in a column of data. You can then map any non-standard value to one of the standard values. The result is a column of consistent data.

= Libraries included with Analytica =

The following libraries are copied to your computer when you install any edition of Analytica (including Trial). You can conveniently add these to a model by selecting '''Add Library...''' from the '''File''' menu in Analyica.

== Bayes Function ==

Functions for a simple application of Bayes' rule to a complete discrete joint array of prior probabilities.

Download: [[Image:Bayes Function.ana]]

== Complex Numbers ==

Functions for computing with complex numbers.

Download: [[Image:Complex Library.ana]]

== Concatenation ==

Functions for concatenation of arrays. Several ConcatN functions allow N arrays to be concatenated in a single call, as an alternative to calling the built-in Concat function N times in sequence. ConcatRows joins the rows of an array .

Download: [[Image:Concatenation.ana]]

== Data Statistics ==

This library is deprecated (obsolete) in Analytica 4.0, but is still included for backward compatibility for models that used these functions in earlier versions of Analytica. Functions allow statistics, such as SDevation, to be applied across indexes other than Run. The built-in statistical functions now allow the running index to be optionally specified.

Download: [[Image:Data Statistics Library.ana]]

== [[Distribution Densities Library|Distribution Densities]] ==

Functions that return the analytic density function for continuous distributions. These densities can be used when using importance sampling, for example. The library is described in detail at [[Distribution Densities Library]].

A new version (after the 4.1.1 release) is now available that includes density functions for all built-in continuous and discrete distributions, as well as for a few other distributions (F-dist, Wishart, Gaussian,...). This version also include cumulative distribution functions and inverse cumulative distribution functions for many distributions. See [[Distribution Densities Library]] for more details. To get the newest version, download the library below, then replace the one in your Libraries directory.

Download: [[media:Distribution Densities.ana|Distribution Densities.ana]]

== Distribution Variations ==

A variety of alternative parametrizations of built-in univariate distributions, and other common parametric univariate distributions that are simple transformations of built-in distributions.

Download: [[Image:Distribution Variations.ana]]

== Expand Index ==
Download: [[Image:Expand Index.ana]]

== Financial Functions ==

Includes functions for derivative valuations, bond pricing, etc.

Note: Standard financial functions (equivalent to those found in Excel) are built into Analytica and are found in the built-in Financial Functions library.

Download: [[Image:Financial Library.ana]]

== Flat File Library ==

Functions for reading data from a comma-separated flat file format (.CSV) into Analytica arrays.

Download: [[Image:Flat File Library.ana]]

== Garbage Bin ==

Provides a garbage bin icon. Instead of deleting nodes, you can drag them to the garbage bin. The "deletion" can then be undone if necessary. Also, if the node you delete has dependents or dependencies, this is immediately evident by the arrows emanating to or from the garbage bin.

To empty the garbage bin, double click on it to open it, select all and delete. Emptying the garbage bin in this fashion is not undoable. You must delete an object (permanently) from the garbage bin before you can reuse its identifier.

Download: [[Image:Garbage Bin.ana]]

== Generalized Regression ==

(Requires Analytica Optimizer )

Logistic and Probit regression. These functions are used to fit a function that predicts the probability of an outcome based on a set of continuous independent variables, from a data set where each data point is classified as either 0 or 1.

Download: [[Image:Generalized Regression.ana]]

== Linked Lists ==

Download: [[Image:Linked List Library.ana]]

== Multivariate Distributions ==

Variout multi-dimensional distributions, and the ability to generated correlated distributions.

Download: [[Image:Multivariate Distributions.ana]]

== ODBC Database Functions ==
* [[Image:ODBC-Library.ana]]

== Optimization Functions ==

Provides some elementary optimization functions:
* GoalSeek
* Solve
* Gradient

These are usable even without Analytica Optimizer; however they are based on a rudimentary gradient-descent algorithm and don't have anything near the power of the Frontline solver functions provided by Analytica Optimizer functions.

Download: [[Image:Optimization Functions.ana]]

== Performance Profiler ==

(Required Analytica Enterprise)

Used to determine where a model is consuming time and memory resources.

Download: [[Image:Performance Profiler.ana]]

Additional libraries

2012-09-06T17:24:17Z

Psanford: /* Libraries Not Included with the Analytica Installer */

(Up to [[Analytica Wiki]])

= Individual functions with examples =

* [[media:Recumulate example.ana|Recumulate example.ana]] : Like [[Cumulate]], but resets to zero at selected points.

= Libraries Not Included with the Analytica Installer =

== [[Large Sample Library: User Guide|Large Sample Library]] ==

[[image:Large sample library.png]]

The Large Sample Library is an Analytica library that lets you run a Monte Carlo simulation for large models or a large sample size that might otherwise exhaust computer memory, including virtual memory. It breaks up a large sample into a series of batch samples, each small enough to run in memory. For selected variables, known as the Large Sample Variables or LSVs, it accumulates the batches into a large sample. You can then view the probability distributions for each LSV using the standard methods — confidence bands, PDF, CDF, etc. — with the full precision of the large sample.

See [[Large Sample Library: User Guide]].

== [[The Sensitivity Analysis Library]] ==

[[The Sensitivity Analysis Library]] provides functions for analyzing the sensitivity of an output to each cell of each array-valued chance input, and locating those individual scalar inputs that have the greatest impact on the result. See [[The Sensitivity Analysis Library]] for documentation on using this library.

The library itself can be downloaded from [[media:Sensitivity Analysis Library.ana|Sensitivity Analysis Library.ana]], and an example model to demonstrate its usage is at [[media:Sensitivity Functions Examples.ana|Sensitivity Functions Examples.ana]].

== Model Documentation Library ==

'''Download: ''' [[media:Model Documentation Library.ana|Model Documentation Library.ana]]

'''Requires:''' Analytica 4.1.0 or later

Some people like to create a concise report on paper that contains all objects in their model along with descriptions or other selected attributes. The "Print Report" feature in Analytica can be used for this purpose, but is not at all concise and ends up placing each object window on a separate page. The ''Model Documentation Library'' allows you to select a module from your model and produce a result table containing every object within that module with its selected attributes such as title, description, units, definition, or user-defined attributes. This table can then be exported to Excel where you can format it nicely and print it. Thus, you can end up with a very concise report on paper.

To use this library, load your model and then select '''File-->Add Module...'''. Add the [[media:Model Documentation Library.ana|Model Documentation Library.ana]] module file using Embed. In that module, using the pulldown, select the top-level module for the report. Follow the instructions shown on the diagram. If you make changes to your model later, press the ''Update Model Documentation'' button to adjust the pulldown content.

== Units Conversion Library ==

'''Download: ''' [[media:Units conversion library.ana|Units conversion library.ana]]

The functions in this library provide conversions between different units -- for example, from feet to kilometers, or from Btu (British thermal units) to gigajoules of energy. Using this library relieves you from having to look up conversion factors. It also makes the model more transparent by making it clear where you are converting from one units to another -- instead of just embedding conversion constants in the formulas.

Use Conversion_Factor(oldUnits, newUnits) to give a conversion factor between two units.
If you omit newUnits, it will assume the standard units for that type (dimensions).
When specifying units , you must use a symbol (abbreviation) from the Units table.

If the units are of different dimensions, e.g. energy and power, or if it doesn't recognize the units, it will give an error.

Use Nom_to_real_dollars(yr, baseYr) to convert from nominal dollars for year yr to "real dollars" for specified baseYr. If you omit baseYr, it defaults to the Standard base year, initially 2000.

These conversion factors were provided on October 20, 2007 by Chris Marnay of the Lawrence Berkeley Laboratory, as part of his work on the SEDS project.

== Greatest Common Divisor functions ==

'''Download: ''' [[media:GCD function library.ana|GCD function library.ana]]

This library contains two [[User-Defined Functions]] for computing the greatest common divisor.

== DB Conversion Library ==

'''Download: ''' [[media:Db conversion lib.ana|DB conversion lib.ana]]

Press a button in this library to transform all the variables defined using [[DbQuery]], [[DbLabels]] and [[DbTable]] to literal data. Variables and indexes defined using [[DbQuery]] or [[DbLabels]] are transformed to list definitions, and those defined using [[DbTable]] are transformed into edit tables. The transformation has the effect of breaking the connection to the external database.

The library is limited in its scope. It only works when all calls to [[DbQuery]], [[DbLabels]] and [[DbTable]] occur at the top level of variable nodes. Do not attempt to use it if you have calls to these functions embedded within larger expressions or within [[User-Defined Functions]].

The purpose of this library is to make it possible to send a copy of your model to another person (including to Lumina Tech Support) who does not have access to your external database, or who does not have Analytica Enterprise.

'''''Use with extreme caution!!''''': Make a copy of your original model ''before'' adding and executing this module. After running the transformation, be sure to use '''File→Save As...''' to save the transformed model under a different filename, so you don't clobber your original model.

== Data Standardization Library ==

'''Download: ''' [[media:Data Standardization Library.ana|Data Standardization Library.ana]]

Imported data is often inconsistent. This library allows choose what the "standard" values should be in a column of data. You can then map any non-standard value to one of the standard values. The result is a column of consistent data.

= Libraries included with Analytica =

The following libraries are copied to your computer when you install any edition of Analytica (including Trial). You can conveniently add these to a model by selecting '''Add Library...''' from the '''File''' menu in Analyica.

== Bayes Function ==

Functions for a simple application of Bayes' rule to a complete discrete joint array of prior probabilities.

Download: [[Image:Bayes Function.ana]]

== Complex Numbers ==

Functions for computing with complex numbers.

Download: [[Image:Complex Library.ana]]

== Concatenation ==

Functions for concatenation of arrays. Several ConcatN functions allow N arrays to be concatenated in a single call, as an alternative to calling the built-in Concat function N times in sequence. ConcatRows joins the rows of an array .

Download: [[Image:Concatenation.ana]]

== Data Statistics ==

This library is deprecated (obsolete) in Analytica 4.0, but is still included for backward compatibility for models that used these functions in earlier versions of Analytica. Functions allow statistics, such as SDevation, to be applied across indexes other than Run. The built-in statistical functions now allow the running index to be optionally specified.

Download: [[Image:Data Statistics Library.ana]]

== [[Distribution Densities Library|Distribution Densities]] ==

Functions that return the analytic density function for continuous distributions. These densities can be used when using importance sampling, for example. The library is described in detail at [[Distribution Densities Library]].

A new version (after the 4.1.1 release) is now available that includes density functions for all built-in continuous and discrete distributions, as well as for a few other distributions (F-dist, Wishart, Gaussian,...). This version also include cumulative distribution functions and inverse cumulative distribution functions for many distributions. See [[Distribution Densities Library]] for more details. To get the newest version, download the library below, then replace the one in your Libraries directory.

Download: [[media:Distribution Densities.ana|Distribution Densities.ana]]

== Distribution Variations ==

A variety of alternative parametrizations of built-in univariate distributions, and other common parametric univariate distributions that are simple transformations of built-in distributions.

Download: [[Image:Distribution Variations.ana]]

== Expand Index ==
Download: [[Image:Expand Index.ana]]

== Financial Functions ==

Includes functions for derivative valuations, bond pricing, etc.

Note: Standard financial functions (equivalent to those found in Excel) are built into Analytica and are found in the built-in Financial Functions library.

Download: [[Image:Financial Library.ana]]

== Flat File Library ==

Functions for reading data from a comma-separated flat file format (.CSV) into Analytica arrays.

Download: [[Image:Flat File Library.ana]]

== Garbage Bin ==

Provides a garbage bin icon. Instead of deleting nodes, you can drag them to the garbage bin. The "deletion" can then be undone if necessary. Also, if the node you delete has dependents or dependencies, this is immediately evident by the arrows emanating to or from the garbage bin.

To empty the garbage bin, double click on it to open it, select all and delete. Emptying the garbage bin in this fashion is not undoable. You must delete an object (permanently) from the garbage bin before you can reuse its identifier.

Download: [[Image:Garbage Bin.ana]]

== Generalized Regression ==

(Requires Analytica Optimizer )

Logistic and Probit regression. These functions are used to fit a function that predicts the probability of an outcome based on a set of continuous independent variables, from a data set where each data point is classified as either 0 or 1.

Download: [[Image:Generalized Regression.ana]]

== Linked Lists ==

Download: [[Image:Linked List Library.ana]]

== Multivariate Distributions ==

Variout multi-dimensional distributions, and the ability to generated correlated distributions.

Download: [[Image:Multivariate Distributions.ana]]

== ODBC Database Functions ==
* [[Image:ODBC-Library.ana]]

== Optimization Functions ==

Provides some elementary optimization functions:
* GoalSeek
* Solve
* Gradient

These are usable even without Analytica Optimizer; however they are based on a rudimentary gradient-descent algorithm and don't have anything near the power of the Frontline solver functions provided by Analytica Optimizer functions.

Download: [[Image:Optimization Functions.ana]]

== Performance Profiler ==

(Required Analytica Enterprise)

Used to determine where a model is consuming time and memory resources.

Download: [[Image:Performance Profiler.ana]]

Regular Expressions

2011-07-19T00:18:57Z

Psanford: /* Basics of Regular Expression */

[[Category:Concepts]]
[[Category:Text Functions]]

Regular expressions are a concise and powerful, but cryptic, formalism for identifying patterns of text to match. They can be quite useful for parsing text files that have minor variability in their formats. They play a prominent role in several programming languages, most notably Perl and Python.

Starting with release 4.2, Analytica provides very powerful (Perl-compatible) regular expression processing within several of its built-in [[:Category:Text Functions|text functions]], notably [[FindInText]], [[SplitText]], and [[TextReplace]]. Each of these functions takes a pattern, which is interpreted as a regular expression when you also specify an optional parameter: ''re:True''. For example:

{To find the position of a seven-letter word:}
[[FindInText]]("\b\w{7}\b","Now is the time for all good men to come to the aid of their country",re:1) → 62

{Split on any word having two repeated letters,}
[[SplitText]]("When in the course of human events, it becomes necessary for ...","[^\w]*\b\w*(\w)\w*\1\w*\b[^\w]*",re:1)→
["When in the course of human", "it", "", "for ..."]

= Basics of Regular Expression =

Regular expressions consist of basic uninterpreted characters (such as the letters and digits), and several special characters that are interpreted. A simple sequence of non-special characters, like "this", is a simple regular expression that matches when that precise sequence of characters occurs anywhere within the subject text.

The power of regular expressions comes from the special sequences that can be used to specify large classes of matching patterns. For example, the dot character means ''match any character'', so that the regular expression "t..s" matches anywhere a "t" is followed by any two characters and then by "s", such as "this", "ttts", "t as", etc. If you want to match only to a dot in the subject text, then you preceed the dot with a backslash, e.g., "t.\." matches "th." and "ts." but not "ths". In general, if you want to use any of the special characters as literals, you preceed them with a backslash.

Other special characters are these:

<pre>
\ general escape character with several uses
^ assert start of string (or line, in multiline mode)
$ assert end of string (or line, in multiline mode)
. match any character except newline (by default)
[ start character class definition
| start of alternative branch
( start subpattern
) end subpattern
? extends the meaning of (
also 0 or 1 quantifier
also quantifier minimizer
* 0 or more quantifier
+ 1 or more quantifier
also "possessive quantifier"
{ start min/max quantifier
\Q...\E Treat all characters between \Q and \E as literals
</pre>
Part of a pattern that is in square brackets is called a "character class". In
a character class the only metacharacters are:
<pre>
\ general escape character
^ negate the class, but only if the first character
- indicates character range
[ POSIX character class (only if followed by POSIX syntax)
] terminates the character class
</pre>

You can refer to several non-printing characters using the following sequences:
<pre>
\a alarm, that is, the BEL character (hex 07)
\cx "control-x", where x is any character
\e escape (hex 1B)
\f formfeed (hex 0C)
\n linefeed (hex 0A)
\r carriage return (hex 0D)
\t tab (hex 09)
\ddd character with octal code ddd, or backreference
\xhh character with hex code hh
\x{hhh..} character with hex code hhh..
</pre>

Several character groups have special escape sequences, including:
<pre>
\w Match a "word" character (letters plus "_")
\W Match a non-"word" character
\s Match a whitespace character
\S Match a non-whitespace character
\d Match a digit character
\D Match a non-digit character
</pre>

And several escape characters match particular points within text that correspond to a position but not to an actual character:
<pre>
\b matches at a word boundary
\B matches when not at a word boundary
\A matches at the start of the subject
\Z matches at the end of the subject
also matches before a newline at the end of the subject
\z matches only at the end of the subject
\G matches at the first matching position in the subject
</pre>

The full specification of regular expression patterns supported is described at [http://www.newlisp.org/downloads/pcrepattern.html Pcre Patterns Man Page].

= Multi-line matching =

By default, a regular expression can be used to match over multiple lines in the source text. The caret (^) and dollar ($) patterns match only the very first and very last character in the entire text, and don't match to the first character in a particular line. The \r and \n patterns can, of course, be used to match to line breaks (in most cases within Analytica, lines will be terminated with \r, but you aren't sure which type of line break you are dealing with, you can always use <code>(?:\r\n)|\r|\n</code>.

You can instruct the matcher to operate in a multi-line mode, in which the text is treated as if composed of separate lines, where a pattern exists on a single line. In this mode, caret (^) matches each line start and dollar ($) matches each line end. To use this mode, begin the regular expression with <code>(?m)</code>.

In theory (according to the Pcre library documenation), you should be able to control which newline character combinations are recognized as the beginning and end of the line. We haven't seen this work, so it may not actually have an effect. To indicate that any newline character combination should be recognized, start the regular expression with <code>(*ANY)</code>, as in: "(*ANY)^\w\d{5}" (which would match to a line within the text beginning with a letter and 5 digits). The (*ANY) prefix considers any standard new-line combination (CR, LF, CRLF) to denote a line break.

Three conventions exist for new lines in text file formats. CR is the standard on the Mac. LF is standard on Unix. CRLF (two characters) is the standard in Windows. Analytica's functions like [ReadTextFile] typically convert to just CR. Excel on Windows (and in CSV files) may use CR for new-rows and LF for new-lines within a single cell. So, depending on where your data is coming from, there are sometimes cases in which you may want to use a multi-line mode, but only with a particular new-line character or combination recognized. The (*ANY) prefix recognizes any of these standard conventions as denoting a newline. (*CR) recognizes only CR, (*LF) recognizes on LF, and (*CRLF) recognizes only the CRLF combination. Note that each of these is a prefix that puts the matcher into a multi-line mode -- the character combinations (*CR) would not appear within the regular expression.

= Finding Patterns in Text =

The [[FindInText]] function, with several optional parameters, can be used to find patterns in text.

====[[FindInText]](pattern, text'', caseInsensitive, re, return, subPattern'')====
* ''pattern'': the regular expression
* ''text'': the subject text being searched
* ''caseInsensitive'': When set to 1, matches 'a' to 'A', etc. Matches are case-sensitive by default.
* ''re'': Must be non-zero for pattern to be interpreted as a regular expression.
* ''return'': Specifies what information should be returned, as follows:
** 'P' (or 'Position'): The position in the subject ''text'' where the matched pattern was found, or zero if not found.
** 'L' (or 'Length'): The length of the match in the subject text.
** 'S' (or 'SubPattern'): The subtext matched by the pattern
** '#' (or '#SubPatterns'): The number of subpatterns in the regular expression.
* ''subPattern'': Which subpattern to return information on. See below.

When using [[FindInText]], you have four different options for what information can be returned. By default, the position of the match (or zero if there is no match) is returned, but alternatively you can have it return the length of the match or the actual text that was successfully matched to . For example:

:[[FindInText]]("[an]+", "A banana in a cabana", re:1, return:'S') → "anana"

If you want to obtain multiple items of information (such as the position, location and matching text) all at the same time, without repeating the match, pass an array to the ''return'' parameter.

== Subpatterns ==

You can group subpatterns in a regular expression using parentheses. You can then extract the values matches to a particular subpattern by specifying which subpattern you are interested in using the ''subPattern'' parameter. The zeroth subpattern always corresponds to the full pattern, and from there grouped expressions are numbered in a depth-first order. You can also specify a group using parentheses whose contents is not to be retained using ''(?:...)''

For example:

Index I := 0..4;
[[FindInText]]("([\w_]+)\s*:\s*((\d*,){4})(\d*),", "NodeInfo: 1,1,1,1,1,1,0,,0,", re:1, return:'S', subPattern:I)
→
{| border="1"
! 0 || "NodeInfo : 1,1,1,1,1,"
|-
! 1 || "NodeInfo"
|-
! 2 || "1,1,1,1,"
|-
! 3 || "1,"
|-
! 4 || "1"
|}

You can see here that ''subPattern:4'' in this example extracts the 5th number in the comma-separated list.

To figure out how many subPatterns are present, you can set the ''return'' parameter to '#'. If ''return'' contains only '#' (i.e., it isn't an array with other 'P', 'L' or 'S' elements), it will determine the number of subPatterns in the regular expression without actually executing a matching search. Thus, if you wanted to pass an index to ''subPattern'', you can figure out how long to make the index before executing the match.

There can be many groupings, and the number and order of groups may change as you debug your regular expression, so using numbered subpatterns is not always the best. You can instead use named subpatterns. The syntax for naming a group is: (?<name>...), or (?'name'....) or (?P<name>...). When you have named a subpattern, you can extract its value by passing the textual name to the ''subPattern'' parameter.

[[FindInText]]("([\w_]+)\s*:\s*((\d*,){4})(?<border>\d*),",
"NodeInfo: 1,1,1,1,1,1,0,,0,", re:1,
return:'S', subPattern:'border') → "1"

== Duplicate Subpatterns ==

Cases frequently arise in which there are two or more alternative syntaxes for subpattern, requiring two subpatterns within the regular expression to have the same name, but usually these are disjunctive. For example, in a standard Excel-compatible CSV format, a cell with no comma or new-line characters does not need to be quoted, but if the cell contains a comma, quotes must be placed around it. For example, a line of a CSV file might be:

:San Jose,"1,006,102",10,Chuck Reed,"SJ,San José,SJC"

This CSV entry has 5 items separated by commas, but two items have internal commas and thus are quoted. Thus, each item matches one of two possible regular expressions, either: <code>([^,]+)</code> or <code>"(.*?)"</code>. Notice that the parenthesis in the second case do not include the quotes, since we do not which to include that in the pattern. To match either, we form a disjunction, but since they refer to the item, we name both branches with the same subpattern name:

("(?<city>.+?)")|(?<city>[^,]+?),\s*("(?<pop>.+?)")|(?<pop>[^,]+?)

Because the two subpatterns named ''city'' are disjunctive, only one of them will match. So, when you request the subpattern "city", you'll get the one which matched (the second in the example). Similarly, only one "pop" subpattern will match, in this example the first, so you'll get info for the one that actually matched.

You could have multiple matches to a subpattern (either named or numbered), as occurs with the regular expression "b(a)*c" applied to "dbaaacd". There is a limitation here in that you can only get the data for one of the repeated matches, the last one.

= Splitting on a Pattern =

You can provide a regular expression as the separator to the [[SplitText]] function. This makes it possible to split text into parts in such a way that allows multiple types of separators, variable length separators, or uncertainty about what the separator will be.

For example, to split on any punctuation character:
[[SplitText]]( text, "[\.\?,!]", re:1 )

Or to split on any number of spaces, so that you don't get blank spaces between separators:

[[SplitText]](text, "\s+", re:1 )

Notice that the parameter ''re:1'' must be specified to cause the separator to be interpreted as a regular expression.

= Substitutions =

The [[TextReplace]] function accepts a regular expression as its pattern when the ''re:1'' parameter is specified.

====[[TextReplace]](text,pattern,subst'',all,caseInsensitive,re)====

* ''text'' : the subject text being matched to
* ''pattern'': The regular expression
* ''subst'' : the text to be substituted for the subtext that matches pattern
* ''all'' : 0=replace only the first occurrence (default), 1=replace every occurrence
* ''caseInsensitive'': 1='A' matches 'a', etc. CaseSensitive by default.
* ''re'': Must be set to 1 for regular expressions

It is recommended that you use a named-parameter calling syntax for the optional parameters. Here are some examples:

:TextReplace("3.141592654", "1|5|9", "0", re:1 ) → "3.041592564"
:TextReplace("3.141592654", "1|5|9", "0", re:1, all:1 ) → "3.040002604"
:TextReplace("3.141592654", "(1|5|9)+", "0", re:1, all:1 ) → "3.140002654"

== SubPattern Substitutions ==

When regular expressions are used, the ''subst'' parameter may refer to subPattern groupings that appear in the ''pattern'' parameter. The matching text for those is substituted accordingly. ''\0'' denotes the full text matched by the full regular expression, ''\1'' is the first subpattern, ''\2'' the second, up to ''\9''.

You can also refer to named subpatterns using ''<name>'' in the ''subst'' parameter. Again, the subtext matching the corresponding named subpattern is substituted. Some examples:

:TextReplace("3.141592", "(\d)", "\1\1", re:1 ) → "33.141592"
:TextReplace("3.141592", "(\d)", "\1\1", re:1, all:1 ) → "33.114411559922"
:TextReplace("time", "(.)(.)(.)(.)", "\4\3\2\1", re:1, all:1 ) → "emit"
:TextReplace("543,632","(?<x>\d+),(?<y>\d+)", "<y>,<x>", re:1, all:1) → "632,543"

= Credits =

Analytica makes use of the ''Perl Compatible Regular Expression'' library, written by Philip Hazel (email: ph10 at cam.ac.uk) of the University of Cambridge Computing Service, Cambridge, England.
Copyright (c) 1997-2008 University of Cambridge
All rights reserved.

The library is included in Analytica under the "BSD" license published with the PCRE release 7.8 distributable.

Regular Expressions

2011-07-19T00:01:23Z

Psanford: /* Basics of Regular Expression */

[[Category:Concepts]]
[[Category:Text Functions]]

Regular expressions are a concise and powerful, but cryptic, formalism for identifying patterns of text to match. They can be quite useful for parsing text files that have minor variability in their formats. They play a prominent role in several programming languages, most notably Perl and Python.

Starting with release 4.2, Analytica provides very powerful (Perl-compatible) regular expression processing within several of its built-in [[:Category:Text Functions|text functions]], notably [[FindInText]], [[SplitText]], and [[TextReplace]]. Each of these functions takes a pattern, which is interpreted as a regular expression when you also specify an optional parameter: ''re:True''. For example:

{To find the position of a seven-letter word:}
[[FindInText]]("\b\w{7}\b","Now is the time for all good men to come to the aid of their country",re:1) → 62

{Split on any word having two repeated letters,}
[[SplitText]]("When in the course of human events, it becomes necessary for ...","[^\w]*\b\w*(\w)\w*\1\w*\b[^\w]*",re:1)→
["When in the course of human", "it", "", "for ..."]

= Basics of Regular Expression =

Regular expressions consist of basic uninterpreted characters (such as the letters and digits), and several special characters that are interpreted. A simple sequence of non-special characters, like "this", is a simple regular expression that matches when that precise sequence of characters occurs anywhere within the subject text.

The power of regular expressions comes from the special sequences that can be used to specify large classes of matching patterns. For example, the dot character means ''match any character'', so that the regular expression "t..s" matches anywhere a "t" is followed by any two characters and then by "s", such as "this", "ttts", "t as", etc. If you want to match only to a dot in the subject text, then you preceed the dot with a backslash, e.g., "t.\." matches "th." and "ts." but not "ths". In general, if you want to use any of the special characters as literals, you preceed them with a backslash.

Other special characters are these:

<pre>
\ general escape character with several uses
^ assert start of string (or line, in multiline mode)
$ assert end of string (or line, in multiline mode)
. match any character except newline (by default)
[ start character class definition
| start of alternative branch
( start subpattern
) end subpattern
? extends the meaning of (
also 0 or 1 quantifier
also quantifier minimizer
* 0 or more quantifier
+ 1 or more quantifier
also "possessive quantifier"
{ start min/max quantifier
</pre>
Part of a pattern that is in square brackets is called a "character class". In
a character class the only metacharacters are:
<pre>
\ general escape character
^ negate the class, but only if the first character
- indicates character range
[ POSIX character class (only if followed by POSIX syntax)
] terminates the character class
</pre>

You can refer to several non-printing characters using the following sequences:
<pre>
\a alarm, that is, the BEL character (hex 07)
\cx "control-x", where x is any character
\e escape (hex 1B)
\f formfeed (hex 0C)
\n linefeed (hex 0A)
\r carriage return (hex 0D)
\t tab (hex 09)
\ddd character with octal code ddd, or backreference
\xhh character with hex code hh
\x{hhh..} character with hex code hhh..
</pre>

Several character groups have special escape sequences, including:
<pre>
\w Match a "word" character (letters plus "_")
\W Match a non-"word" character
\s Match a whitespace character
\S Match a non-whitespace character
\d Match a digit character
\D Match a non-digit character
\Q...\E Treat all characters between \Q and \E as literals
</pre>

And several escape characters match particular points within text that correspond to a position but not to an actual character:
<pre>
\b matches at a word boundary
\B matches when not at a word boundary
\A matches at the start of the subject
\Z matches at the end of the subject
also matches before a newline at the end of the subject
\z matches only at the end of the subject
\G matches at the first matching position in the subject
</pre>

The full specification of regular expression patterns supported is described at [http://www.newlisp.org/downloads/pcrepattern.html Pcre Patterns Man Page].

= Multi-line matching =

By default, a regular expression can be used to match over multiple lines in the source text. The caret (^) and dollar ($) patterns match only the very first and very last character in the entire text, and don't match to the first character in a particular line. The \r and \n patterns can, of course, be used to match to line breaks (in most cases within Analytica, lines will be terminated with \r, but you aren't sure which type of line break you are dealing with, you can always use <code>(?:\r\n)|\r|\n</code>.

You can instruct the matcher to operate in a multi-line mode, in which the text is treated as if composed of separate lines, where a pattern exists on a single line. In this mode, caret (^) matches each line start and dollar ($) matches each line end. To use this mode, begin the regular expression with <code>(?m)</code>.

In theory (according to the Pcre library documenation), you should be able to control which newline character combinations are recognized as the beginning and end of the line. We haven't seen this work, so it may not actually have an effect. To indicate that any newline character combination should be recognized, start the regular expression with <code>(*ANY)</code>, as in: "(*ANY)^\w\d{5}" (which would match to a line within the text beginning with a letter and 5 digits). The (*ANY) prefix considers any standard new-line combination (CR, LF, CRLF) to denote a line break.

Three conventions exist for new lines in text file formats. CR is the standard on the Mac. LF is standard on Unix. CRLF (two characters) is the standard in Windows. Analytica's functions like [ReadTextFile] typically convert to just CR. Excel on Windows (and in CSV files) may use CR for new-rows and LF for new-lines within a single cell. So, depending on where your data is coming from, there are sometimes cases in which you may want to use a multi-line mode, but only with a particular new-line character or combination recognized. The (*ANY) prefix recognizes any of these standard conventions as denoting a newline. (*CR) recognizes only CR, (*LF) recognizes on LF, and (*CRLF) recognizes only the CRLF combination. Note that each of these is a prefix that puts the matcher into a multi-line mode -- the character combinations (*CR) would not appear within the regular expression.

= Finding Patterns in Text =

The [[FindInText]] function, with several optional parameters, can be used to find patterns in text.

====[[FindInText]](pattern, text'', caseInsensitive, re, return, subPattern'')====
* ''pattern'': the regular expression
* ''text'': the subject text being searched
* ''caseInsensitive'': When set to 1, matches 'a' to 'A', etc. Matches are case-sensitive by default.
* ''re'': Must be non-zero for pattern to be interpreted as a regular expression.
* ''return'': Specifies what information should be returned, as follows:
** 'P' (or 'Position'): The position in the subject ''text'' where the matched pattern was found, or zero if not found.
** 'L' (or 'Length'): The length of the match in the subject text.
** 'S' (or 'SubPattern'): The subtext matched by the pattern
** '#' (or '#SubPatterns'): The number of subpatterns in the regular expression.
* ''subPattern'': Which subpattern to return information on. See below.

When using [[FindInText]], you have four different options for what information can be returned. By default, the position of the match (or zero if there is no match) is returned, but alternatively you can have it return the length of the match or the actual text that was successfully matched to . For example:

:[[FindInText]]("[an]+", "A banana in a cabana", re:1, return:'S') → "anana"

If you want to obtain multiple items of information (such as the position, location and matching text) all at the same time, without repeating the match, pass an array to the ''return'' parameter.

== Subpatterns ==

You can group subpatterns in a regular expression using parentheses. You can then extract the values matches to a particular subpattern by specifying which subpattern you are interested in using the ''subPattern'' parameter. The zeroth subpattern always corresponds to the full pattern, and from there grouped expressions are numbered in a depth-first order. You can also specify a group using parentheses whose contents is not to be retained using ''(?:...)''

For example:

Index I := 0..4;
[[FindInText]]("([\w_]+)\s*:\s*((\d*,){4})(\d*),", "NodeInfo: 1,1,1,1,1,1,0,,0,", re:1, return:'S', subPattern:I)
→
{| border="1"
! 0 || "NodeInfo : 1,1,1,1,1,"
|-
! 1 || "NodeInfo"
|-
! 2 || "1,1,1,1,"
|-
! 3 || "1,"
|-
! 4 || "1"
|}

You can see here that ''subPattern:4'' in this example extracts the 5th number in the comma-separated list.

To figure out how many subPatterns are present, you can set the ''return'' parameter to '#'. If ''return'' contains only '#' (i.e., it isn't an array with other 'P', 'L' or 'S' elements), it will determine the number of subPatterns in the regular expression without actually executing a matching search. Thus, if you wanted to pass an index to ''subPattern'', you can figure out how long to make the index before executing the match.

There can be many groupings, and the number and order of groups may change as you debug your regular expression, so using numbered subpatterns is not always the best. You can instead use named subpatterns. The syntax for naming a group is: (?<name>...), or (?'name'....) or (?P<name>...). When you have named a subpattern, you can extract its value by passing the textual name to the ''subPattern'' parameter.

[[FindInText]]("([\w_]+)\s*:\s*((\d*,){4})(?<border>\d*),",
"NodeInfo: 1,1,1,1,1,1,0,,0,", re:1,
return:'S', subPattern:'border') → "1"

== Duplicate Subpatterns ==

Cases frequently arise in which there are two or more alternative syntaxes for subpattern, requiring two subpatterns within the regular expression to have the same name, but usually these are disjunctive. For example, in a standard Excel-compatible CSV format, a cell with no comma or new-line characters does not need to be quoted, but if the cell contains a comma, quotes must be placed around it. For example, a line of a CSV file might be:

:San Jose,"1,006,102",10,Chuck Reed,"SJ,San José,SJC"

This CSV entry has 5 items separated by commas, but two items have internal commas and thus are quoted. Thus, each item matches one of two possible regular expressions, either: <code>([^,]+)</code> or <code>"(.*?)"</code>. Notice that the parenthesis in the second case do not include the quotes, since we do not which to include that in the pattern. To match either, we form a disjunction, but since they refer to the item, we name both branches with the same subpattern name:

("(?<city>.+?)")|(?<city>[^,]+?),\s*("(?<pop>.+?)")|(?<pop>[^,]+?)

Because the two subpatterns named ''city'' are disjunctive, only one of them will match. So, when you request the subpattern "city", you'll get the one which matched (the second in the example). Similarly, only one "pop" subpattern will match, in this example the first, so you'll get info for the one that actually matched.

You could have multiple matches to a subpattern (either named or numbered), as occurs with the regular expression "b(a)*c" applied to "dbaaacd". There is a limitation here in that you can only get the data for one of the repeated matches, the last one.

= Splitting on a Pattern =

You can provide a regular expression as the separator to the [[SplitText]] function. This makes it possible to split text into parts in such a way that allows multiple types of separators, variable length separators, or uncertainty about what the separator will be.

For example, to split on any punctuation character:
[[SplitText]]( text, "[\.\?,!]", re:1 )

Or to split on any number of spaces, so that you don't get blank spaces between separators:

[[SplitText]](text, "\s+", re:1 )

Notice that the parameter ''re:1'' must be specified to cause the separator to be interpreted as a regular expression.

= Substitutions =

The [[TextReplace]] function accepts a regular expression as its pattern when the ''re:1'' parameter is specified.

====[[TextReplace]](text,pattern,subst'',all,caseInsensitive,re)====

* ''text'' : the subject text being matched to
* ''pattern'': The regular expression
* ''subst'' : the text to be substituted for the subtext that matches pattern
* ''all'' : 0=replace only the first occurrence (default), 1=replace every occurrence
* ''caseInsensitive'': 1='A' matches 'a', etc. CaseSensitive by default.
* ''re'': Must be set to 1 for regular expressions

It is recommended that you use a named-parameter calling syntax for the optional parameters. Here are some examples:

:TextReplace("3.141592654", "1|5|9", "0", re:1 ) → "3.041592564"
:TextReplace("3.141592654", "1|5|9", "0", re:1, all:1 ) → "3.040002604"
:TextReplace("3.141592654", "(1|5|9)+", "0", re:1, all:1 ) → "3.140002654"

== SubPattern Substitutions ==

When regular expressions are used, the ''subst'' parameter may refer to subPattern groupings that appear in the ''pattern'' parameter. The matching text for those is substituted accordingly. ''\0'' denotes the full text matched by the full regular expression, ''\1'' is the first subpattern, ''\2'' the second, up to ''\9''.

You can also refer to named subpatterns using ''<name>'' in the ''subst'' parameter. Again, the subtext matching the corresponding named subpattern is substituted. Some examples:

:TextReplace("3.141592", "(\d)", "\1\1", re:1 ) → "33.141592"
:TextReplace("3.141592", "(\d)", "\1\1", re:1, all:1 ) → "33.114411559922"
:TextReplace("time", "(.)(.)(.)(.)", "\4\3\2\1", re:1, all:1 ) → "emit"
:TextReplace("543,632","(?<x>\d+),(?<y>\d+)", "<y>,<x>", re:1, all:1) → "632,543"

= Credits =

Analytica makes use of the ''Perl Compatible Regular Expression'' library, written by Philip Hazel (email: ph10 at cam.ac.uk) of the University of Cambridge Computing Service, Cambridge, England.
Copyright (c) 1997-2008 University of Cambridge
All rights reserved.

The library is included in Analytica under the "BSD" license published with the PCRE release 7.8 distributable.

Subset

2011-05-31T20:40:42Z

Psanford: /* Examples */

[[Category:Functions that create lists]]
[[Category:Doc Status C]] 

= Subset(D'', position, I, resultIndex, warnEmpty'') =

Returns a list containing all the index elements of the index of one-D array for which «D» is true.

(new in [[Analytica 4.2]]) The optional «position» parameter specifies whether the index position or index element should be returned. When omitted or when <code>position:false</code> is specified, the index element is returned. <code>Position:true</code> returns the index position of true items.

(new in [[Analytica 4.2]]) The optional index parameters «I» and «resultIndex» allow [[Subset]] to be used in a manner that fully array abstractions. Normally, without these, «D» must be one-dimensional and will not evaluate or array abstract otherwise (since an unindexed list is returned). «I» specifies which dimension to operate over, and «resultIndex» specifies an index to dimension the result by. When these are used in conjunction, the result is an array with fully explicit indexes. When you are using [[Subset]] to define an index, you should not be using «resultIndex», «D» should be 1-dimensional, and «I» is superfluous. When you are using it in calculation that is potentially array-based, then you should be using «I» and «resultIndex».

(new in [[Analytica 4.2]]) The optional «warnEmpty» parameter controls whether [[Subset]] issues a warning when no true elements are found in «D». In Analytica 4.2 and earlier this defaulted to True, in Analytica 4.3 and later it defaults to false.

= When to use =

Use [[Subset]] to create a new index that is a subset of an existing index.

= Variations =

When there is the possibility of having [[Null]] values in your array, you should consider whether you want those elements to be retained. The following two uses respectively retain or remove [[Null]] elements:
[[Subset]](Test [[Or]] Test=[[Null]])
[[Subset]](Test [[And]] Test<>[[Null]])
Without the explicit test, [[Null]] elements are removed in release 4.2 and later, but retained prior to 4.2.

(''new to release 4.2'') The position (rather than the index value) of each non-zero element can be obtained using:
[[Subset]](D,position:true)
See [[Associative vs. Positional Indexing]].

= Examples =

[[Subset]](Years < 1987) → [1985,1986]

[[Subset]](Years < 1987, position:true) → [1,2] { ''requires 4.2'' }

= Array Considerations =

== Implicit indexes ==

The one-dimensional array passed to [[Subset]] needs to be explicitly indexed, since the index elements corresponding to true elements is returned. Thus, an implicitly indexed array (e.g., a list) also results in an error. Theoretically [[Subset]] could return a result in the implicitly indexed case, but it would be misleading since it would just be returning all the true values, for example:
[[Subset]]( 1997..2008 > 2005 ) →? [1,1,1] { Hypothetical only -- error reported }
In a case like this, you should use a local index, e.g.:
[[Index..Do|Index]] I := 1997..2008 do [[Subset]](I>2005) → [2006,2007,2008]

However, if you use the ''position:true'' parameter setting, then you can pass an array with an implicit index, since element positions (and not index values) are returned. Hence, the following does work (in ''Analytica 4.2 or better''):
[[Subset]]( 1997..2008 > 2005, position:true ) → [10,11,12]

== Array abstraction ==

An error will result if the parameter to [[Subset]] is not one-dimensional. [[Subset]] in its standard one-parameter usage does not array abstract. A scalar parameter or a parameter with two or more dimensions results in an error.

[[What's new in Analytica 4.2?|Analytica 4.2]] provides a variation of [[Subset]] that does array abstract. If you have a two- (or more-) dimensional array, you'll need to identify one of those dimensions as the one that the subset is being taken over. For each slice along the other dimensions, a [[Subset]] can then be separately computed, and each of those subsets may have a different number of elements. To express the result, you must provide a «resultIndex» that will contain the index result. For the individual subsets that contain more elements than the result index, only the first [[Size]](«resultIndex») members of the subset are retained. For those subsets that are smaller than the «resultIndex», the remaining entries in the result are padded with [[«null»]] values.

Example:
:{| border="1"
! rowspan=2 colspan=2 | A
! colspan=5 align=center | I
|-
! 1 !! 2 !! 3 !! 4 !! 5
|-
| rowspan=3 | J
! 1
| 73 || 23 || 19 || 54 || 12
|-
! 2
| 11 || 1 || 98 || 76 || 52
|-
! 3
| 22 || 33 || 49 || 87 || 22
|}

:Index K := [1,2]

:{|border="0" style="text-align:center"
| [[Subset]](A>50,I:I,ResultIndex:K) →
|
{| border="1"
! rowspan=2 colspan=2 |  
! colspan=2 align=center | K
|-
! width="50px" | 1 !! width="50px" | 2
|-
| rowspan=3 | J
! 1
| 1 || 4
|-
! 2
| 3 || 4
|-
! 3
| 4 || «null»
|}
|}

Notice that in the J=1 for, there are exactly two elements satisfying the condition, [1,4]. In the second row there are three elements, [3,4,5], but since our result index ''K'' has only two elements, we retain only the first two. In the third row of ''A'', there is only one cell satisfying the condition, hence the second cell in the result is padded with «null».

For [[Subset]] to reliably array abstract, both indexes must be specified, although it is fine to use the same index for both. Since they are optional, and not the second and third parameter, they should be specified using the named calling convention as illustrated above.

= See Also =

* [[media:Abstracted Subset.ana|Abstracted Subset.ana]] example model
* [[Associative vs. Positional Indexing]]
* [[SubIndex]]
* [[Subscript]]
* [[Index..Do]]

Example Models

2011-04-23T02:43:10Z

Psanford: /* Linearizing a discrete NSP */

[[category:Doc Status D]]

The Wiki pages here provide a repository for Analytica models and libraries. Supplementary material may be included here describing the model, its usage, etc. Models or libraries may be contributed because they are useful for particular applications, provide a starting point for certain modeling tasks, demonstrate an Analytica concept, etc.

Several dozen models are included with the Analytica distribution, installed onto your machine when you install Analytica. These models are not also here on the Wiki yet, but may be added in the future. Furthermore, as updates to these models occur, more recent versions will be made available here.

Analytica users may also contribute their own models and examples here. For instructions on how to upload your own contributions, see [[Uploading Example Models]].

----
= Timber Post Compression Load Capacity =

Author: Lonnie Chrisman

Here is a calculator for computing the maximum load that can be handled by a Douglas Fir - Larch post of a given size, grade, and composition in a construction setting: [[Media:PostCompression.ana|Post Compression Model]]

----
= Transforming Dimensions by transform matrix, month to qtr =

Author: Lonnie Chrisman

Model: [[Media:Month to quarter.ana|Month to quarter.ana]]

The shows how to transform an array from a finer-grain index (e.g., Month) onto a coarser index (e.g., Quarter). We generally refer to this as [[Aggregate|aggregation]]. The example was introduced when Analytica 4.1 was the current release, but since the addition of the [[Aggregate]] function in [[What's new in Analytica 4.2?|Analytica 4.2]], this transformation is very straightforward. The model has been updated to reflect both methods -- the direct use of [[Aggregate]] in Analytica 4.2, as well as the previous method.

For additional reference material, a webinar exists on using [[Analytica_User_Group/Past_Topics#The_Aggregate_Function|the Aggregate function]].

= Items within Budget function =

Author: Max Henrion

Given a set of items, with a priority and a cost for each, the function Items_within_budget function selects out the highest priority items that fit within the fixed budget. The function is available from: [[Media:Items_within_budget.ana|Items within budget]]

----
= Convolution =

Author: Lonnie Chrisman

The model [[Media:Convolution.ana|Convolution]] contains a function, Convolve(Y,Z,T,I), that computes the convolution of two time series.

A time series is a set of points, (Y,T), where T is the ascending X-axis, and the set of points is indexed by I. The values of T do not have to be equally spaced. The function treats Y and Z as being equal to 0 outside the range of T. The two time series here are the set of points (Y,T) and the set of points (Z,T), where both sets of points are indexed by I.

The model contains a couple examples of convolved functions.

The mathematical definition of the convolution of two time series is the function given by:

h(t) = Integral y(u) z(t-u) dt

Convolution is used predominantly in signal and systems analysis.

----
= Sampling from only feasible points =

Author: Lonnie Chrisman

Consider this scenario. You have a bunch of chance variables, each defined by a distribution. They joint sample generated, however, contains some combinations of points that are (for one reason or another) physically impossible. We'll call those infeasible points. You'd like to eliminate those points from the sample and keep only the feasible points.

The module [[Media:Feasible_Sampler.ana|Feasible Sampler]] implements a button that will sample a collection of chance variables, then reset the sample size and keep only those sample points that are "feasible".

Obviously, this approach will work best when most of your samples are feasible. If you can handle the "infeasible" points in your model directly, by conditioning certain chance variables on others, that is far preferable. But there are certainly cases where this solution (although a bit of a kludge) is more readily usable.

The instructions for how to use this are in the module description field.

----
= Grant Exclusion Model =

[[Media:Grant_exclusion.ANA|Grant Exclusion]]

This model tests a hypothesis about the distribution of an attribute of the marginal rejectee of a grant program, given the relevance of that attribute to award of the grant. It could be used by an organization to make decisions as to whether to fiscally-sponsor another organization that will use that fiscal sponsorship to apply for grants, by looking at the effect on the pool of grant recipients overall.

----
= Donor/Presenter Dashboard =

[[Media:Donor_Presenter_Dashboard_II.ANA|Donor-Presenter Dashboard]]

This model implements a continuous-time Markov chain in Analytica's discrete-time dynamic simulation environment. It supports immigration to, and emigration from, every node.

It can be used by an arts organization to probabilistically forecast future audience evolution, in both the short and the long (steady state) term. It also allows for uncertainty in the input parameters.

----

= Project Planner =

'''Download:''' [[Media:Project Priorities 5 0.ana|Project Priorities 5 0.ana]]

A demo model that shows how to:
* Evaluate a set of R&D projects, including uncertain R&D costs, and uncertain revenues if it leads to release of a commercial product.
* Use multiattribute analysis to compare projects, including a hard attribute -- expected net present value -- and soft attributes -- strategic fit, staff development, and public good will.
* Compare cost, NPV, and multiattribute value for a selected portfolio of projects.
* Generate the best portfolio (ratio of NPV or multiattribute merit to cost) given a R&D budget.

[[Image: Project planner model.png |500px]]

This link is only a test, and to an older version:
<link
target="blank">http://lumina.com/wiki/images/4/43/Project_priorities_2007_4.0.ANA</link>

----

= California Power Plants =

A model that demonstrates the use of [[Choice|choice pulldowns]] in edit tables. The model is created during a mini-tutorial on [[Inserting_Choice_Controls_in_Edit_Table_Cells]] elsewhere on this Wiki.

[[Media:California_Power_Plants.ANA]]

----

= Dependency Tracker Module =

This module tracks dependencies through your model, updating the visual appearance of nodes so that you can quickly visualize the paths by which one variable influences another. You can also use it to provide a visual indication of which nodes are downstream (or upstream) from an indicated variable.

The module contains button scripts that change the bevel appearance of nodes in your model. To see how Variable X influences Variable Y, the script will bevel the nodes for all variables that are influenced by X and influence Y. Alternatively, you can bevel all nodes that are influenced by X, or you can bevel all nodes that influence Y.

[[image:Dependency tracker.jpg]]

In the image above, the path from dp_ex_2 through dp_ex_4 has been highlight using the bevel style of the nodes. (The result of pressing the "Bevel all from Ancestor to Descendant" button)

[[media:Dependency_Tracker.ANA | Dependency_Tracker.ana]]

----

= Total Allowable Harvest =

The problem applies to any population of fish or animal whose dynamics are poorly known but can be summarized in a simple model:

N_t+1 = N_t*Lambda - landed catch*(1+loss rate)

where N_t is the population size (number of individuals) at time t, N_t+1 is the population size at time t+1, Lambda is the intrinsic rate of increase and the loss rate is the percentage of fish or animals killed but not retrieved relative to the landed catch, or catch secured.

The question here is to determine how many fish or animals can be caught (landed) annually so that the probability of the population declining X% in Y years (decline threshold) is less than Z% (risk tolerance).

Two models are available for download. One uses the Optimizer ([[NlpDefine]]) to find the maximum landed catch at the risk tolerance level for the given decline threshold. The other (for those using a version of Analytica without Optimizer) uses [[StepInterp]] in an iterative way to get that maximum landed catch.

* [[media:Total Allowable Removal model with Optimizer.ana | Total Allowable w Optimizer.ana]]
* [[media:Total Allowable Removal model w StepInterp.ana|Total Allowable w StepInterp.ana]]

Models contributed by Pierre Richard

----

= Earthquake Expenses =

An example of risk analysis with time-dependence and costs shifted over time.

Certain organizations (insurance companies, large companies, governments) incur expenses following earthquakes. This simplified demo model can be used to answer questions such as:
* What is the probability of more than one quake in a specific 10 year period.
* What is the probability that in my time window my costs exceed $X?

Assumptions in this model:
* Earthquakes are Poisson events with mean rate of once every 10 years.
* Damage caused by such quake is lognormally distributed, with mean $10M adn stddev of $6M.
* Cost of damage gets incurred over the period of a year from the date of the quake as equipment is replaced and buildings are repaired over time: 20% in 1st quarter after quake, 50% in 2nd quarter, 20% in 3rd quarter, 10% in 4th quarter.

Model file: [[media:Earthquake expenses.ana|Earthquake expenses.ana]]

----

= Regulation of Photosynthesis =

Author Lonnie Chrisman, Ph.D.

[[image:Photosynthesis fluorescence.jpg]]

A model of how photosynthesis is regulated inside a cyanobacteria. As light exposure varies over time (and you can experiment with various light intensity waveforms), it simulates the concentration levels of key transport molecules along the chain, through the PSII complex, plasto-quinone pool, PSI complex, down to metabolic oxidation. The dynamic response to light levels, or changes in light levels, over time becomes evident, and the impact of changes to metabolic demand can also be observed. In the graph of fluorescence above, we can see an indicator of how much energy is being absorbed, in three different cases (different light intensities). In the two higher intensity cases, photoinhibition is observed -- a protective mechanism of the cell that engages when more energy is coming in than can be utilized by the cell. Excess incoming energy, in the absense of photoinhibition, causes damage, particularly to the PSII complex.

This model uses node shapes for a different purpose than is normally seen in decision analysis models. In this model, ovals, instead of depicting chance variables, depict chemical reactions, where the value depicts the reaction rate, and rounded rectangles depict chemical concentrations.

Two models are attached. The first is a bit cleaner, and focused on the core transport chain, as described above. The second is less developed, but is focused more on genetic regulation processes.

* [[media:Photosynthesis regulation.ANA|Photosynthesis Regulation.ana]] - main regulation pathways
* [[media:Photosystem.ana | Photosystem.ana]] - rough sketch of genetic regulation.

----

= Cross-Validation / Fitting Kernel Functions to Data =

Author: Lonnie Chrisman, Ph.D., Lumina Decision Systems

[[image:Cross-validated data fit.jpg]]

Model: [[media:Cross-validation example.ana|Cross-validation example.ana]]

When fitting a function to data, if you have too many free parameters relative to the number of points in your data set, you may "overfit" the data. When this happens, the fit to your training data may be very good, but the fit to new data points (beyond those used for training) may be very poor.

Cross-validation is a common technique to deal with this problem. With this technique, we set aside a fraction of the available data as a cross-validation set. Then we begin by fitting very simple functions to the data (with very few free parameters), successively increasing the number of free parameters, and seeing how the predictive performance changes on the cross-validation set. It is typical to see improvement on the cross-validation set for a while, followed by a deterioriation of predictive performance on the cross-validation set once overfitting starts occuring.

This example model successively fits a non-linear kernel function to the residual error, and uses cross-validation to determine how many kernel functions should be used.

Requires Analytica Optimizer: The kernel fitting function (Kern_Fit) uses [[NlpDefine]].

----
= Statistical Bootstrapping =

Model: [[media:Bootstrapping.ana|Bootstrapping.ana]]

Bootstrapping is a technique from statistics for estimating the sampling error present in a statistical estimator. The simplest version estimates sampling error by resampling the original data. This model demonstrates how this is accomplished in Analytica.

----

= Compression Post Load Calculator =

Author: Lonnie Chrisman

Model: [[media:Compression_Post_Load_Capacity.ana|Compression_Post_Load_Capacity.ana]]

Computes the load that a Douglas-Fir Larch post can support in compression. Works for different timber types and grades and post sizes.

----
= Daylighting Options in Building Design =

Author: Max Henrion

Model: [[media:Daylighting analyzer.ana|Daylighting analyzer.ana]]

A demonstration showing how to analyze lifecycle costs and savings from daylighting options in building design.

Analysis based on Nomograph Cost/Benefit Tool for Daylighting. adapted from S.E. Selkowitz and M. Gabel. 1984. "LBL Daylighting Nomographs," LBL-13534, Lawrence Berkeley Laboratory, Berkeley CA, 94704. (510) 486-6845.

----
= Plane Catching Decision with EVIU =

Author: Max Henrion

Model: [[media:Plane catching decision with EVIU.ana|Plane catching decision with EVIU.ana]]

A simple model to assess what time I should leave my home to catch a plane, with uncertain driving time, walking from parking to gate (including security), and how long I need to be at the gate ahead of scheduled departure time. It uses a loss model based on minutes, assuming I value each extra minute snoozing in bed and set the loss if I miss the plane to 400 of those minutes.

It illustrates the EVIU (expected value of including uncertainty) i.e. the difference in expected value if I make a decision to minimize expected loss instead of decision to minimize time ignoring uncertainty (assuming each distribution is fixed at its mid value). For more details see "The Value of Knowing How Little You Know", Max Henrion, PhD Dissertation, Carnegie Mellon University, 1982.

----

= Marginal Analysis for Control of SO2 emissions =

Author: Surya Swamy

Model: [[media:Marginal Analysis for Control of SO2 Emissions.ana|Marginal Analysis for Control of SO2 Emissions.ana]]

Acid rain in eastern US and Canada caused by sulfur dioxide is emitted primarily by coal-burning electric-generating plants in the Midwestern U.S. This model demonstrates a marginal analysis a.k.a. benefit/cost analysis to determine the policy alternative that leads us to the most economically efficient level of cleanup.

----

= Electrical Generation and Transmission =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Electrical Transmission.ana|Electrical Transmission.ana]]

''Requires Analytica Optimizer''

This is a simple model of an electrical distribution network. At each node in the network we have power generators and power consumers (demand). Nodes are connected by branches (i.e., transmission lines), where each branch has a given admittance (the real part of impedeance is assumed to be zero) and a maximum capacity in Watts. Each power generator has a min and max generation capability with a given marginal rate per kilowatt-hour. The model uses a linear program to determine how much power each generator should produce so as to minimize total production cost, while satisfying demand and remaining within branch capacity constraints.

----
= Loan Policy Selection =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Loan policy selection.ANA|Loan policy selection.ANA]]

'''Best used with Analytica Optimizer'''

A lender has a large pool of money to loan, but needs to decide what credit rating threshold to require and what interest rate (above prime) to charge. The optimal value is determined by market forces (competiting lenders) and by the probability that the borrower defaults on the loan, which is a function of the economy and borrower's credit rating. The model can be used without the Analytica optimizer, in which case you can explore the decision space manually or use a parametric analysis to find the near optimal solution. Those with Analytica Optimizer can find the optimal solution (more quickly) using an [[NlpDefine|NLP]] search.
----

= Time-series re-indexing =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Time-series-reindexing.ana|Time-series-reindexing.ana]]

This model contains some examples of time-series re-indexing. It is intended to demonstrate some of these basic techniques.

[[image:Weekly_data_graph_ex.png]]

In this example, actual measurements were collected at non-uniform time increments. Before analyzing these, we map these to a uniformly spaced time index (Week), occuring on Monday of each week. The mapping is done using an interpolation. The evenly-spaced data is then used to forecast future behavior. We first forecast over an index containing only future time points (Future_weeks), using a log-normal process model based on the historical weekly change. We then combine the historical data with the forecast on a common index (Week). A prob-bands graph of the weekly_data result shows the range of uncertainty projected by the process model (you'll notice the uncertainty exists only for future forecasted values, not historical ones).

----

= Multi-lingual Influence Diagram =

'''Author:''' D. Rice, Lumina Decision Systems.

'''Model:''' [[media:French-English.ana|French-English.ana]]

{| border="0"
| [[Image:English-view.png]]
| [[Image:French-view.png]]
|}

Maintains a single influence diagram with Title and Description attributes in both English and French. With the change of a pull-down, the influence diagram and all object descriptions are instantly reflected in the language of choice.

If you change a title or description while viewing English, and then change to French, the change you made will become the English-language version of the description. Similarly if you make a change while viewing French.

= Smooth PDF plots using Kernel Density Estimation =

'''Author:''' D. Rice, Lumina Decision Systems

'''Model:''' [[media:Kernel_Density_Estimation.ana|Kernel_Density_Estimation.ana]]

{| border="0"
| [[image:Dens_Est_builtin_pdf.png|frame|Analytica's built-in PDF plot with default settings]]
|
[[image:Dens_Est_Kernel_pdf.png|frame|PDF computed from Kernel Density estimation]]
|}

This example demonstrates a very simple fixed-width kernel density estimator to estimate a "smooth" probability density. The built-in PDF function in Analytica often has a choppy appearance due to the nature of histogramming -- it sets up a set of bins and counts how many points land in each bin. A kernel density estimator smooths this out, producing a less choppy PDF plot.

= Output and Input Columns in Same Table =

'''Author:''' D. Rice, Lumina Decision Systems

'''Model:''' [[media:Output and input columns.ana|Output and input columns.ana]]

[[image:Output and input columns.png]]

Presents an input table to a user, where one column is populated with computed output data, the other column with checkboxes for the user to select. Although the '''Output Data''' column isn't read only, as would be desired, a [[Check Attribute]] has been configured to complain if he does try to change values in that column. The model that uses these inputs would ignore any changes he makes to data in the '''Output Data''' column.

Populating the '''Output Data''' column requires the user to press a button, which runs a button script to populate that column. This button is presented on the top-level panel. If you change the input value, the output data will change, and then the button needs to be pressed to refresh the output data column.

= Linearizing a discrete NSP =

'''Author:''' P. Sanford, Lumina Decision Systems

'''Model:''' [[media:Cereal Formulation1.ana|Cereal Formulation1.ana]]

Cereal formulation model

A discrete mixed integer model that chooses product formulations to minimize total ingredient costs. This could be an NSP but it uses two methods to linearize:
1) Decision variable is constructed as a constrained Boolean array
2) Prices are defined as piecewise linear curves

= Neural Network =

'''Author:''' Lonnie Chrisman, Lumina Decision Systems

'''Model:''' [[media:Neural-Network.ana|Neural-Network.ana]]

A feed-forward neural network can be trained (fit to training data) using the Analytica Optimizer. This is essentially an example of non-linear regression. This model contains four sample data sets, and is set up to train a 2-layer feedforward sigmoid network to "learn" the concept represented by the data set(s), and then test how well it does across examples not appearing in the training set.

Developed during the Analytica User Group Webinar of 21-Apr-2011 -- see the [[Analytica_User_Group/Past_Topics#Neural_Networks|webinar recording]].

Example Models

2011-04-23T02:40:08Z

Psanford: /* Linearizing a discrete NSP */

[[category:Doc Status D]]

The Wiki pages here provide a repository for Analytica models and libraries. Supplementary material may be included here describing the model, its usage, etc. Models or libraries may be contributed because they are useful for particular applications, provide a starting point for certain modeling tasks, demonstrate an Analytica concept, etc.

Several dozen models are included with the Analytica distribution, installed onto your machine when you install Analytica. These models are not also here on the Wiki yet, but may be added in the future. Furthermore, as updates to these models occur, more recent versions will be made available here.

Analytica users may also contribute their own models and examples here. For instructions on how to upload your own contributions, see [[Uploading Example Models]].

----
= Timber Post Compression Load Capacity =

Author: Lonnie Chrisman

Here is a calculator for computing the maximum load that can be handled by a Douglas Fir - Larch post of a given size, grade, and composition in a construction setting: [[Media:PostCompression.ana|Post Compression Model]]

----
= Transforming Dimensions by transform matrix, month to qtr =

Author: Lonnie Chrisman

Model: [[Media:Month to quarter.ana|Month to quarter.ana]]

The shows how to transform an array from a finer-grain index (e.g., Month) onto a coarser index (e.g., Quarter). We generally refer to this as [[Aggregate|aggregation]]. The example was introduced when Analytica 4.1 was the current release, but since the addition of the [[Aggregate]] function in [[What's new in Analytica 4.2?|Analytica 4.2]], this transformation is very straightforward. The model has been updated to reflect both methods -- the direct use of [[Aggregate]] in Analytica 4.2, as well as the previous method.

For additional reference material, a webinar exists on using [[Analytica_User_Group/Past_Topics#The_Aggregate_Function|the Aggregate function]].

= Items within Budget function =

Author: Max Henrion

Given a set of items, with a priority and a cost for each, the function Items_within_budget function selects out the highest priority items that fit within the fixed budget. The function is available from: [[Media:Items_within_budget.ana|Items within budget]]

----
= Convolution =

Author: Lonnie Chrisman

The model [[Media:Convolution.ana|Convolution]] contains a function, Convolve(Y,Z,T,I), that computes the convolution of two time series.

A time series is a set of points, (Y,T), where T is the ascending X-axis, and the set of points is indexed by I. The values of T do not have to be equally spaced. The function treats Y and Z as being equal to 0 outside the range of T. The two time series here are the set of points (Y,T) and the set of points (Z,T), where both sets of points are indexed by I.

The model contains a couple examples of convolved functions.

The mathematical definition of the convolution of two time series is the function given by:

h(t) = Integral y(u) z(t-u) dt

Convolution is used predominantly in signal and systems analysis.

----
= Sampling from only feasible points =

Author: Lonnie Chrisman

Consider this scenario. You have a bunch of chance variables, each defined by a distribution. They joint sample generated, however, contains some combinations of points that are (for one reason or another) physically impossible. We'll call those infeasible points. You'd like to eliminate those points from the sample and keep only the feasible points.

The module [[Media:Feasible_Sampler.ana|Feasible Sampler]] implements a button that will sample a collection of chance variables, then reset the sample size and keep only those sample points that are "feasible".

Obviously, this approach will work best when most of your samples are feasible. If you can handle the "infeasible" points in your model directly, by conditioning certain chance variables on others, that is far preferable. But there are certainly cases where this solution (although a bit of a kludge) is more readily usable.

The instructions for how to use this are in the module description field.

----
= Grant Exclusion Model =

[[Media:Grant_exclusion.ANA|Grant Exclusion]]

This model tests a hypothesis about the distribution of an attribute of the marginal rejectee of a grant program, given the relevance of that attribute to award of the grant. It could be used by an organization to make decisions as to whether to fiscally-sponsor another organization that will use that fiscal sponsorship to apply for grants, by looking at the effect on the pool of grant recipients overall.

----
= Donor/Presenter Dashboard =

[[Media:Donor_Presenter_Dashboard_II.ANA|Donor-Presenter Dashboard]]

This model implements a continuous-time Markov chain in Analytica's discrete-time dynamic simulation environment. It supports immigration to, and emigration from, every node.

It can be used by an arts organization to probabilistically forecast future audience evolution, in both the short and the long (steady state) term. It also allows for uncertainty in the input parameters.

----

= Project Planner =

'''Download:''' [[Media:Project Priorities 5 0.ana|Project Priorities 5 0.ana]]

A demo model that shows how to:
* Evaluate a set of R&D projects, including uncertain R&D costs, and uncertain revenues if it leads to release of a commercial product.
* Use multiattribute analysis to compare projects, including a hard attribute -- expected net present value -- and soft attributes -- strategic fit, staff development, and public good will.
* Compare cost, NPV, and multiattribute value for a selected portfolio of projects.
* Generate the best portfolio (ratio of NPV or multiattribute merit to cost) given a R&D budget.

[[Image: Project planner model.png |500px]]

This link is only a test, and to an older version:
<link
target="blank">http://lumina.com/wiki/images/4/43/Project_priorities_2007_4.0.ANA</link>

----

= California Power Plants =

A model that demonstrates the use of [[Choice|choice pulldowns]] in edit tables. The model is created during a mini-tutorial on [[Inserting_Choice_Controls_in_Edit_Table_Cells]] elsewhere on this Wiki.

[[Media:California_Power_Plants.ANA]]

----

= Dependency Tracker Module =

This module tracks dependencies through your model, updating the visual appearance of nodes so that you can quickly visualize the paths by which one variable influences another. You can also use it to provide a visual indication of which nodes are downstream (or upstream) from an indicated variable.

The module contains button scripts that change the bevel appearance of nodes in your model. To see how Variable X influences Variable Y, the script will bevel the nodes for all variables that are influenced by X and influence Y. Alternatively, you can bevel all nodes that are influenced by X, or you can bevel all nodes that influence Y.

[[image:Dependency tracker.jpg]]

In the image above, the path from dp_ex_2 through dp_ex_4 has been highlight using the bevel style of the nodes. (The result of pressing the "Bevel all from Ancestor to Descendant" button)

[[media:Dependency_Tracker.ANA | Dependency_Tracker.ana]]

----

= Total Allowable Harvest =

The problem applies to any population of fish or animal whose dynamics are poorly known but can be summarized in a simple model:

N_t+1 = N_t*Lambda - landed catch*(1+loss rate)

where N_t is the population size (number of individuals) at time t, N_t+1 is the population size at time t+1, Lambda is the intrinsic rate of increase and the loss rate is the percentage of fish or animals killed but not retrieved relative to the landed catch, or catch secured.

The question here is to determine how many fish or animals can be caught (landed) annually so that the probability of the population declining X% in Y years (decline threshold) is less than Z% (risk tolerance).

Two models are available for download. One uses the Optimizer ([[NlpDefine]]) to find the maximum landed catch at the risk tolerance level for the given decline threshold. The other (for those using a version of Analytica without Optimizer) uses [[StepInterp]] in an iterative way to get that maximum landed catch.

* [[media:Total Allowable Removal model with Optimizer.ana | Total Allowable w Optimizer.ana]]
* [[media:Total Allowable Removal model w StepInterp.ana|Total Allowable w StepInterp.ana]]

Models contributed by Pierre Richard

----

= Earthquake Expenses =

An example of risk analysis with time-dependence and costs shifted over time.

Certain organizations (insurance companies, large companies, governments) incur expenses following earthquakes. This simplified demo model can be used to answer questions such as:
* What is the probability of more than one quake in a specific 10 year period.
* What is the probability that in my time window my costs exceed $X?

Assumptions in this model:
* Earthquakes are Poisson events with mean rate of once every 10 years.
* Damage caused by such quake is lognormally distributed, with mean $10M adn stddev of $6M.
* Cost of damage gets incurred over the period of a year from the date of the quake as equipment is replaced and buildings are repaired over time: 20% in 1st quarter after quake, 50% in 2nd quarter, 20% in 3rd quarter, 10% in 4th quarter.

Model file: [[media:Earthquake expenses.ana|Earthquake expenses.ana]]

----

= Regulation of Photosynthesis =

Author Lonnie Chrisman, Ph.D.

[[image:Photosynthesis fluorescence.jpg]]

A model of how photosynthesis is regulated inside a cyanobacteria. As light exposure varies over time (and you can experiment with various light intensity waveforms), it simulates the concentration levels of key transport molecules along the chain, through the PSII complex, plasto-quinone pool, PSI complex, down to metabolic oxidation. The dynamic response to light levels, or changes in light levels, over time becomes evident, and the impact of changes to metabolic demand can also be observed. In the graph of fluorescence above, we can see an indicator of how much energy is being absorbed, in three different cases (different light intensities). In the two higher intensity cases, photoinhibition is observed -- a protective mechanism of the cell that engages when more energy is coming in than can be utilized by the cell. Excess incoming energy, in the absense of photoinhibition, causes damage, particularly to the PSII complex.

This model uses node shapes for a different purpose than is normally seen in decision analysis models. In this model, ovals, instead of depicting chance variables, depict chemical reactions, where the value depicts the reaction rate, and rounded rectangles depict chemical concentrations.

Two models are attached. The first is a bit cleaner, and focused on the core transport chain, as described above. The second is less developed, but is focused more on genetic regulation processes.

* [[media:Photosynthesis regulation.ANA|Photosynthesis Regulation.ana]] - main regulation pathways
* [[media:Photosystem.ana | Photosystem.ana]] - rough sketch of genetic regulation.

----

= Cross-Validation / Fitting Kernel Functions to Data =

Author: Lonnie Chrisman, Ph.D., Lumina Decision Systems

[[image:Cross-validated data fit.jpg]]

Model: [[media:Cross-validation example.ana|Cross-validation example.ana]]

When fitting a function to data, if you have too many free parameters relative to the number of points in your data set, you may "overfit" the data. When this happens, the fit to your training data may be very good, but the fit to new data points (beyond those used for training) may be very poor.

Cross-validation is a common technique to deal with this problem. With this technique, we set aside a fraction of the available data as a cross-validation set. Then we begin by fitting very simple functions to the data (with very few free parameters), successively increasing the number of free parameters, and seeing how the predictive performance changes on the cross-validation set. It is typical to see improvement on the cross-validation set for a while, followed by a deterioriation of predictive performance on the cross-validation set once overfitting starts occuring.

This example model successively fits a non-linear kernel function to the residual error, and uses cross-validation to determine how many kernel functions should be used.

Requires Analytica Optimizer: The kernel fitting function (Kern_Fit) uses [[NlpDefine]].

----
= Statistical Bootstrapping =

Model: [[media:Bootstrapping.ana|Bootstrapping.ana]]

Bootstrapping is a technique from statistics for estimating the sampling error present in a statistical estimator. The simplest version estimates sampling error by resampling the original data. This model demonstrates how this is accomplished in Analytica.

----

= Compression Post Load Calculator =

Author: Lonnie Chrisman

Model: [[media:Compression_Post_Load_Capacity.ana|Compression_Post_Load_Capacity.ana]]

Computes the load that a Douglas-Fir Larch post can support in compression. Works for different timber types and grades and post sizes.

----
= Daylighting Options in Building Design =

Author: Max Henrion

Model: [[media:Daylighting analyzer.ana|Daylighting analyzer.ana]]

A demonstration showing how to analyze lifecycle costs and savings from daylighting options in building design.

Analysis based on Nomograph Cost/Benefit Tool for Daylighting. adapted from S.E. Selkowitz and M. Gabel. 1984. "LBL Daylighting Nomographs," LBL-13534, Lawrence Berkeley Laboratory, Berkeley CA, 94704. (510) 486-6845.

----
= Plane Catching Decision with EVIU =

Author: Max Henrion

Model: [[media:Plane catching decision with EVIU.ana|Plane catching decision with EVIU.ana]]

A simple model to assess what time I should leave my home to catch a plane, with uncertain driving time, walking from parking to gate (including security), and how long I need to be at the gate ahead of scheduled departure time. It uses a loss model based on minutes, assuming I value each extra minute snoozing in bed and set the loss if I miss the plane to 400 of those minutes.

It illustrates the EVIU (expected value of including uncertainty) i.e. the difference in expected value if I make a decision to minimize expected loss instead of decision to minimize time ignoring uncertainty (assuming each distribution is fixed at its mid value). For more details see "The Value of Knowing How Little You Know", Max Henrion, PhD Dissertation, Carnegie Mellon University, 1982.

----

= Marginal Analysis for Control of SO2 emissions =

Author: Surya Swamy

Model: [[media:Marginal Analysis for Control of SO2 Emissions.ana|Marginal Analysis for Control of SO2 Emissions.ana]]

Acid rain in eastern US and Canada caused by sulfur dioxide is emitted primarily by coal-burning electric-generating plants in the Midwestern U.S. This model demonstrates a marginal analysis a.k.a. benefit/cost analysis to determine the policy alternative that leads us to the most economically efficient level of cleanup.

----

= Electrical Generation and Transmission =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Electrical Transmission.ana|Electrical Transmission.ana]]

''Requires Analytica Optimizer''

This is a simple model of an electrical distribution network. At each node in the network we have power generators and power consumers (demand). Nodes are connected by branches (i.e., transmission lines), where each branch has a given admittance (the real part of impedeance is assumed to be zero) and a maximum capacity in Watts. Each power generator has a min and max generation capability with a given marginal rate per kilowatt-hour. The model uses a linear program to determine how much power each generator should produce so as to minimize total production cost, while satisfying demand and remaining within branch capacity constraints.

----
= Loan Policy Selection =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Loan policy selection.ANA|Loan policy selection.ANA]]

'''Best used with Analytica Optimizer'''

A lender has a large pool of money to loan, but needs to decide what credit rating threshold to require and what interest rate (above prime) to charge. The optimal value is determined by market forces (competiting lenders) and by the probability that the borrower defaults on the loan, which is a function of the economy and borrower's credit rating. The model can be used without the Analytica optimizer, in which case you can explore the decision space manually or use a parametric analysis to find the near optimal solution. Those with Analytica Optimizer can find the optimal solution (more quickly) using an [[NlpDefine|NLP]] search.
----

= Time-series re-indexing =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Time-series-reindexing.ana|Time-series-reindexing.ana]]

This model contains some examples of time-series re-indexing. It is intended to demonstrate some of these basic techniques.

[[image:Weekly_data_graph_ex.png]]

In this example, actual measurements were collected at non-uniform time increments. Before analyzing these, we map these to a uniformly spaced time index (Week), occuring on Monday of each week. The mapping is done using an interpolation. The evenly-spaced data is then used to forecast future behavior. We first forecast over an index containing only future time points (Future_weeks), using a log-normal process model based on the historical weekly change. We then combine the historical data with the forecast on a common index (Week). A prob-bands graph of the weekly_data result shows the range of uncertainty projected by the process model (you'll notice the uncertainty exists only for future forecasted values, not historical ones).

----

= Multi-lingual Influence Diagram =

'''Author:''' D. Rice, Lumina Decision Systems.

'''Model:''' [[media:French-English.ana|French-English.ana]]

{| border="0"
| [[Image:English-view.png]]
| [[Image:French-view.png]]
|}

Maintains a single influence diagram with Title and Description attributes in both English and French. With the change of a pull-down, the influence diagram and all object descriptions are instantly reflected in the language of choice.

If you change a title or description while viewing English, and then change to French, the change you made will become the English-language version of the description. Similarly if you make a change while viewing French.

= Smooth PDF plots using Kernel Density Estimation =

'''Author:''' D. Rice, Lumina Decision Systems

'''Model:''' [[media:Kernel_Density_Estimation.ana|Kernel_Density_Estimation.ana]]

{| border="0"
| [[image:Dens_Est_builtin_pdf.png|frame|Analytica's built-in PDF plot with default settings]]
|
[[image:Dens_Est_Kernel_pdf.png|frame|PDF computed from Kernel Density estimation]]
|}

This example demonstrates a very simple fixed-width kernel density estimator to estimate a "smooth" probability density. The built-in PDF function in Analytica often has a choppy appearance due to the nature of histogramming -- it sets up a set of bins and counts how many points land in each bin. A kernel density estimator smooths this out, producing a less choppy PDF plot.

= Output and Input Columns in Same Table =

'''Author:''' D. Rice, Lumina Decision Systems

'''Model:''' [[media:Output and input columns.ana|Output and input columns.ana]]

[[image:Output and input columns.png]]

Presents an input table to a user, where one column is populated with computed output data, the other column with checkboxes for the user to select. Although the '''Output Data''' column isn't read only, as would be desired, a [[Check Attribute]] has been configured to complain if he does try to change values in that column. The model that uses these inputs would ignore any changes he makes to data in the '''Output Data''' column.

Populating the '''Output Data''' column requires the user to press a button, which runs a button script to populate that column. This button is presented on the top-level panel. If you change the input value, the output data will change, and then the button needs to be pressed to refresh the output data column.

= Linearizing a discrete NSP =

'''Author:''' P. Sanford, Lumina Decision Systems

'''Model:''' [[media:Cereal Formulation1.ana|Cereal Formulation1.ana]]

Cereal formulation model

= Neural Network =

'''Author:''' Lonnie Chrisman, Lumina Decision Systems

'''Model:''' [[media:Neural-Network.ana|Neural-Network.ana]]

A feed-forward neural network can be trained (fit to training data) using the Analytica Optimizer. This is essentially an example of non-linear regression. This model contains four sample data sets, and is set up to train a 2-layer feedforward sigmoid network to "learn" the concept represented by the data set(s), and then test how well it does across examples not appearing in the training set.

Developed during the Analytica User Group Webinar of 21-Apr-2011 -- see the [[Analytica_User_Group/Past_Topics#Neural_Networks|webinar recording]].

File:Cereal Formulation1.ana

2011-04-23T02:39:33Z

Psanford: Discrete NSP linearized by using a constrained Boolean matrix. Also uses peicewise linear constraints

Discrete NSP linearized by using a constrained Boolean matrix. Also uses peicewise linear constraints

Example Models

2011-04-23T02:35:53Z

Psanford: /* Linearizing a discrete NSP */

[[category:Doc Status D]]

The Wiki pages here provide a repository for Analytica models and libraries. Supplementary material may be included here describing the model, its usage, etc. Models or libraries may be contributed because they are useful for particular applications, provide a starting point for certain modeling tasks, demonstrate an Analytica concept, etc.

Several dozen models are included with the Analytica distribution, installed onto your machine when you install Analytica. These models are not also here on the Wiki yet, but may be added in the future. Furthermore, as updates to these models occur, more recent versions will be made available here.

Analytica users may also contribute their own models and examples here. For instructions on how to upload your own contributions, see [[Uploading Example Models]].

----
= Timber Post Compression Load Capacity =

Author: Lonnie Chrisman

Here is a calculator for computing the maximum load that can be handled by a Douglas Fir - Larch post of a given size, grade, and composition in a construction setting: [[Media:PostCompression.ana|Post Compression Model]]

----
= Transforming Dimensions by transform matrix, month to qtr =

Author: Lonnie Chrisman

Model: [[Media:Month to quarter.ana|Month to quarter.ana]]

The shows how to transform an array from a finer-grain index (e.g., Month) onto a coarser index (e.g., Quarter). We generally refer to this as [[Aggregate|aggregation]]. The example was introduced when Analytica 4.1 was the current release, but since the addition of the [[Aggregate]] function in [[What's new in Analytica 4.2?|Analytica 4.2]], this transformation is very straightforward. The model has been updated to reflect both methods -- the direct use of [[Aggregate]] in Analytica 4.2, as well as the previous method.

For additional reference material, a webinar exists on using [[Analytica_User_Group/Past_Topics#The_Aggregate_Function|the Aggregate function]].

= Items within Budget function =

Author: Max Henrion

Given a set of items, with a priority and a cost for each, the function Items_within_budget function selects out the highest priority items that fit within the fixed budget. The function is available from: [[Media:Items_within_budget.ana|Items within budget]]

----
= Convolution =

Author: Lonnie Chrisman

The model [[Media:Convolution.ana|Convolution]] contains a function, Convolve(Y,Z,T,I), that computes the convolution of two time series.

A time series is a set of points, (Y,T), where T is the ascending X-axis, and the set of points is indexed by I. The values of T do not have to be equally spaced. The function treats Y and Z as being equal to 0 outside the range of T. The two time series here are the set of points (Y,T) and the set of points (Z,T), where both sets of points are indexed by I.

The model contains a couple examples of convolved functions.

The mathematical definition of the convolution of two time series is the function given by:

h(t) = Integral y(u) z(t-u) dt

Convolution is used predominantly in signal and systems analysis.

----
= Sampling from only feasible points =

Author: Lonnie Chrisman

Consider this scenario. You have a bunch of chance variables, each defined by a distribution. They joint sample generated, however, contains some combinations of points that are (for one reason or another) physically impossible. We'll call those infeasible points. You'd like to eliminate those points from the sample and keep only the feasible points.

The module [[Media:Feasible_Sampler.ana|Feasible Sampler]] implements a button that will sample a collection of chance variables, then reset the sample size and keep only those sample points that are "feasible".

Obviously, this approach will work best when most of your samples are feasible. If you can handle the "infeasible" points in your model directly, by conditioning certain chance variables on others, that is far preferable. But there are certainly cases where this solution (although a bit of a kludge) is more readily usable.

The instructions for how to use this are in the module description field.

----
= Grant Exclusion Model =

[[Media:Grant_exclusion.ANA|Grant Exclusion]]

This model tests a hypothesis about the distribution of an attribute of the marginal rejectee of a grant program, given the relevance of that attribute to award of the grant. It could be used by an organization to make decisions as to whether to fiscally-sponsor another organization that will use that fiscal sponsorship to apply for grants, by looking at the effect on the pool of grant recipients overall.

----
= Donor/Presenter Dashboard =

[[Media:Donor_Presenter_Dashboard_II.ANA|Donor-Presenter Dashboard]]

This model implements a continuous-time Markov chain in Analytica's discrete-time dynamic simulation environment. It supports immigration to, and emigration from, every node.

It can be used by an arts organization to probabilistically forecast future audience evolution, in both the short and the long (steady state) term. It also allows for uncertainty in the input parameters.

----

= Project Planner =

'''Download:''' [[Media:Project Priorities 5 0.ana|Project Priorities 5 0.ana]]

A demo model that shows how to:
* Evaluate a set of R&D projects, including uncertain R&D costs, and uncertain revenues if it leads to release of a commercial product.
* Use multiattribute analysis to compare projects, including a hard attribute -- expected net present value -- and soft attributes -- strategic fit, staff development, and public good will.
* Compare cost, NPV, and multiattribute value for a selected portfolio of projects.
* Generate the best portfolio (ratio of NPV or multiattribute merit to cost) given a R&D budget.

[[Image: Project planner model.png |500px]]

This link is only a test, and to an older version:
<link
target="blank">http://lumina.com/wiki/images/4/43/Project_priorities_2007_4.0.ANA</link>

----

= California Power Plants =

A model that demonstrates the use of [[Choice|choice pulldowns]] in edit tables. The model is created during a mini-tutorial on [[Inserting_Choice_Controls_in_Edit_Table_Cells]] elsewhere on this Wiki.

[[Media:California_Power_Plants.ANA]]

----

= Dependency Tracker Module =

This module tracks dependencies through your model, updating the visual appearance of nodes so that you can quickly visualize the paths by which one variable influences another. You can also use it to provide a visual indication of which nodes are downstream (or upstream) from an indicated variable.

The module contains button scripts that change the bevel appearance of nodes in your model. To see how Variable X influences Variable Y, the script will bevel the nodes for all variables that are influenced by X and influence Y. Alternatively, you can bevel all nodes that are influenced by X, or you can bevel all nodes that influence Y.

[[image:Dependency tracker.jpg]]

In the image above, the path from dp_ex_2 through dp_ex_4 has been highlight using the bevel style of the nodes. (The result of pressing the "Bevel all from Ancestor to Descendant" button)

[[media:Dependency_Tracker.ANA | Dependency_Tracker.ana]]

----

= Total Allowable Harvest =

The problem applies to any population of fish or animal whose dynamics are poorly known but can be summarized in a simple model:

N_t+1 = N_t*Lambda - landed catch*(1+loss rate)

where N_t is the population size (number of individuals) at time t, N_t+1 is the population size at time t+1, Lambda is the intrinsic rate of increase and the loss rate is the percentage of fish or animals killed but not retrieved relative to the landed catch, or catch secured.

The question here is to determine how many fish or animals can be caught (landed) annually so that the probability of the population declining X% in Y years (decline threshold) is less than Z% (risk tolerance).

Two models are available for download. One uses the Optimizer ([[NlpDefine]]) to find the maximum landed catch at the risk tolerance level for the given decline threshold. The other (for those using a version of Analytica without Optimizer) uses [[StepInterp]] in an iterative way to get that maximum landed catch.

* [[media:Total Allowable Removal model with Optimizer.ana | Total Allowable w Optimizer.ana]]
* [[media:Total Allowable Removal model w StepInterp.ana|Total Allowable w StepInterp.ana]]

Models contributed by Pierre Richard

----

= Earthquake Expenses =

An example of risk analysis with time-dependence and costs shifted over time.

Certain organizations (insurance companies, large companies, governments) incur expenses following earthquakes. This simplified demo model can be used to answer questions such as:
* What is the probability of more than one quake in a specific 10 year period.
* What is the probability that in my time window my costs exceed $X?

Assumptions in this model:
* Earthquakes are Poisson events with mean rate of once every 10 years.
* Damage caused by such quake is lognormally distributed, with mean $10M adn stddev of $6M.
* Cost of damage gets incurred over the period of a year from the date of the quake as equipment is replaced and buildings are repaired over time: 20% in 1st quarter after quake, 50% in 2nd quarter, 20% in 3rd quarter, 10% in 4th quarter.

Model file: [[media:Earthquake expenses.ana|Earthquake expenses.ana]]

----

= Regulation of Photosynthesis =

Author Lonnie Chrisman, Ph.D.

[[image:Photosynthesis fluorescence.jpg]]

A model of how photosynthesis is regulated inside a cyanobacteria. As light exposure varies over time (and you can experiment with various light intensity waveforms), it simulates the concentration levels of key transport molecules along the chain, through the PSII complex, plasto-quinone pool, PSI complex, down to metabolic oxidation. The dynamic response to light levels, or changes in light levels, over time becomes evident, and the impact of changes to metabolic demand can also be observed. In the graph of fluorescence above, we can see an indicator of how much energy is being absorbed, in three different cases (different light intensities). In the two higher intensity cases, photoinhibition is observed -- a protective mechanism of the cell that engages when more energy is coming in than can be utilized by the cell. Excess incoming energy, in the absense of photoinhibition, causes damage, particularly to the PSII complex.

This model uses node shapes for a different purpose than is normally seen in decision analysis models. In this model, ovals, instead of depicting chance variables, depict chemical reactions, where the value depicts the reaction rate, and rounded rectangles depict chemical concentrations.

Two models are attached. The first is a bit cleaner, and focused on the core transport chain, as described above. The second is less developed, but is focused more on genetic regulation processes.

* [[media:Photosynthesis regulation.ANA|Photosynthesis Regulation.ana]] - main regulation pathways
* [[media:Photosystem.ana | Photosystem.ana]] - rough sketch of genetic regulation.

----

= Cross-Validation / Fitting Kernel Functions to Data =

Author: Lonnie Chrisman, Ph.D., Lumina Decision Systems

[[image:Cross-validated data fit.jpg]]

Model: [[media:Cross-validation example.ana|Cross-validation example.ana]]

When fitting a function to data, if you have too many free parameters relative to the number of points in your data set, you may "overfit" the data. When this happens, the fit to your training data may be very good, but the fit to new data points (beyond those used for training) may be very poor.

Cross-validation is a common technique to deal with this problem. With this technique, we set aside a fraction of the available data as a cross-validation set. Then we begin by fitting very simple functions to the data (with very few free parameters), successively increasing the number of free parameters, and seeing how the predictive performance changes on the cross-validation set. It is typical to see improvement on the cross-validation set for a while, followed by a deterioriation of predictive performance on the cross-validation set once overfitting starts occuring.

This example model successively fits a non-linear kernel function to the residual error, and uses cross-validation to determine how many kernel functions should be used.

Requires Analytica Optimizer: The kernel fitting function (Kern_Fit) uses [[NlpDefine]].

----
= Statistical Bootstrapping =

Model: [[media:Bootstrapping.ana|Bootstrapping.ana]]

Bootstrapping is a technique from statistics for estimating the sampling error present in a statistical estimator. The simplest version estimates sampling error by resampling the original data. This model demonstrates how this is accomplished in Analytica.

----

= Compression Post Load Calculator =

Author: Lonnie Chrisman

Model: [[media:Compression_Post_Load_Capacity.ana|Compression_Post_Load_Capacity.ana]]

Computes the load that a Douglas-Fir Larch post can support in compression. Works for different timber types and grades and post sizes.

----
= Daylighting Options in Building Design =

Author: Max Henrion

Model: [[media:Daylighting analyzer.ana|Daylighting analyzer.ana]]

A demonstration showing how to analyze lifecycle costs and savings from daylighting options in building design.

Analysis based on Nomograph Cost/Benefit Tool for Daylighting. adapted from S.E. Selkowitz and M. Gabel. 1984. "LBL Daylighting Nomographs," LBL-13534, Lawrence Berkeley Laboratory, Berkeley CA, 94704. (510) 486-6845.

----
= Plane Catching Decision with EVIU =

Author: Max Henrion

Model: [[media:Plane catching decision with EVIU.ana|Plane catching decision with EVIU.ana]]

A simple model to assess what time I should leave my home to catch a plane, with uncertain driving time, walking from parking to gate (including security), and how long I need to be at the gate ahead of scheduled departure time. It uses a loss model based on minutes, assuming I value each extra minute snoozing in bed and set the loss if I miss the plane to 400 of those minutes.

It illustrates the EVIU (expected value of including uncertainty) i.e. the difference in expected value if I make a decision to minimize expected loss instead of decision to minimize time ignoring uncertainty (assuming each distribution is fixed at its mid value). For more details see "The Value of Knowing How Little You Know", Max Henrion, PhD Dissertation, Carnegie Mellon University, 1982.

----

= Marginal Analysis for Control of SO2 emissions =

Author: Surya Swamy

Model: [[media:Marginal Analysis for Control of SO2 Emissions.ana|Marginal Analysis for Control of SO2 Emissions.ana]]

Acid rain in eastern US and Canada caused by sulfur dioxide is emitted primarily by coal-burning electric-generating plants in the Midwestern U.S. This model demonstrates a marginal analysis a.k.a. benefit/cost analysis to determine the policy alternative that leads us to the most economically efficient level of cleanup.

----

= Electrical Generation and Transmission =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Electrical Transmission.ana|Electrical Transmission.ana]]

''Requires Analytica Optimizer''

This is a simple model of an electrical distribution network. At each node in the network we have power generators and power consumers (demand). Nodes are connected by branches (i.e., transmission lines), where each branch has a given admittance (the real part of impedeance is assumed to be zero) and a maximum capacity in Watts. Each power generator has a min and max generation capability with a given marginal rate per kilowatt-hour. The model uses a linear program to determine how much power each generator should produce so as to minimize total production cost, while satisfying demand and remaining within branch capacity constraints.

----
= Loan Policy Selection =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Loan policy selection.ANA|Loan policy selection.ANA]]

'''Best used with Analytica Optimizer'''

A lender has a large pool of money to loan, but needs to decide what credit rating threshold to require and what interest rate (above prime) to charge. The optimal value is determined by market forces (competiting lenders) and by the probability that the borrower defaults on the loan, which is a function of the economy and borrower's credit rating. The model can be used without the Analytica optimizer, in which case you can explore the decision space manually or use a parametric analysis to find the near optimal solution. Those with Analytica Optimizer can find the optimal solution (more quickly) using an [[NlpDefine|NLP]] search.
----

= Time-series re-indexing =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Time-series-reindexing.ana|Time-series-reindexing.ana]]

This model contains some examples of time-series re-indexing. It is intended to demonstrate some of these basic techniques.

[[image:Weekly_data_graph_ex.png]]

In this example, actual measurements were collected at non-uniform time increments. Before analyzing these, we map these to a uniformly spaced time index (Week), occuring on Monday of each week. The mapping is done using an interpolation. The evenly-spaced data is then used to forecast future behavior. We first forecast over an index containing only future time points (Future_weeks), using a log-normal process model based on the historical weekly change. We then combine the historical data with the forecast on a common index (Week). A prob-bands graph of the weekly_data result shows the range of uncertainty projected by the process model (you'll notice the uncertainty exists only for future forecasted values, not historical ones).

----

= Multi-lingual Influence Diagram =

'''Author:''' D. Rice, Lumina Decision Systems.

'''Model:''' [[media:French-English.ana|French-English.ana]]

{| border="0"
| [[Image:English-view.png]]
| [[Image:French-view.png]]
|}

Maintains a single influence diagram with Title and Description attributes in both English and French. With the change of a pull-down, the influence diagram and all object descriptions are instantly reflected in the language of choice.

If you change a title or description while viewing English, and then change to French, the change you made will become the English-language version of the description. Similarly if you make a change while viewing French.

= Smooth PDF plots using Kernel Density Estimation =

'''Author:''' D. Rice, Lumina Decision Systems

'''Model:''' [[media:Kernel_Density_Estimation.ana|Kernel_Density_Estimation.ana]]

{| border="0"
| [[image:Dens_Est_builtin_pdf.png|frame|Analytica's built-in PDF plot with default settings]]
|
[[image:Dens_Est_Kernel_pdf.png|frame|PDF computed from Kernel Density estimation]]
|}

This example demonstrates a very simple fixed-width kernel density estimator to estimate a "smooth" probability density. The built-in PDF function in Analytica often has a choppy appearance due to the nature of histogramming -- it sets up a set of bins and counts how many points land in each bin. A kernel density estimator smooths this out, producing a less choppy PDF plot.

= Output and Input Columns in Same Table =

'''Author:''' D. Rice, Lumina Decision Systems

'''Model:''' [[media:Output and input columns.ana|Output and input columns.ana]]

[[image:Output and input columns.png]]

Presents an input table to a user, where one column is populated with computed output data, the other column with checkboxes for the user to select. Although the '''Output Data''' column isn't read only, as would be desired, a [[Check Attribute]] has been configured to complain if he does try to change values in that column. The model that uses these inputs would ignore any changes he makes to data in the '''Output Data''' column.

Populating the '''Output Data''' column requires the user to press a button, which runs a button script to populate that column. This button is presented on the top-level panel. If you change the input value, the output data will change, and then the button needs to be pressed to refresh the output data column.

= Linearizing a discrete NSP =

'''Author:''' P. Sanford, Lumina Decision Systems

'''Model:''' [[media:Cereal Formulation.ana|Cereal Formulation.ana]]

Cereal formulation model

= Neural Network =

'''Author:''' Lonnie Chrisman, Lumina Decision Systems

'''Model:''' [[media:Neural-Network.ana|Neural-Network.ana]]

A feed-forward neural network can be trained (fit to training data) using the Analytica Optimizer. This is essentially an example of non-linear regression. This model contains four sample data sets, and is set up to train a 2-layer feedforward sigmoid network to "learn" the concept represented by the data set(s), and then test how well it does across examples not appearing in the training set.

Developed during the Analytica User Group Webinar of 21-Apr-2011 -- see the [[Analytica_User_Group/Past_Topics#Neural_Networks|webinar recording]].

Example Models

2011-04-20T21:28:09Z

Psanford: /* Output and Input Columns in Same Table */

[[category:Doc Status D]]

The Wiki pages here provide a repository for Analytica models and libraries. Supplementary material may be included here describing the model, its usage, etc. Models or libraries may be contributed because they are useful for particular applications, provide a starting point for certain modeling tasks, demonstrate an Analytica concept, etc.

Several dozen models are included with the Analytica distribution, installed onto your machine when you install Analytica. These models are not also here on the Wiki yet, but may be added in the future. Furthermore, as updates to these models occur, more recent versions will be made available here.

Analytica users may also contribute their own models and examples here. For instructions on how to upload your own contributions, see [[Uploading Example Models]].

----
= Timber Post Compression Load Capacity =

Author: Lonnie Chrisman

Here is a calculator for computing the maximum load that can be handled by a Douglas Fir - Larch post of a given size, grade, and composition in a construction setting: [[Media:PostCompression.ana|Post Compression Model]]

----
= Transforming Dimensions by transform matrix, month to qtr =

Author: Lonnie Chrisman

Model: [[Media:Month to quarter.ana|Month to quarter.ana]]

The shows how to transform an array from a finer-grain index (e.g., Month) onto a coarser index (e.g., Quarter). We generally refer to this as [[Aggregate|aggregation]]. The example was introduced when Analytica 4.1 was the current release, but since the addition of the [[Aggregate]] function in [[What's new in Analytica 4.2?|Analytica 4.2]], this transformation is very straightforward. The model has been updated to reflect both methods -- the direct use of [[Aggregate]] in Analytica 4.2, as well as the previous method.

For additional reference material, a webinar exists on using [[Analytica_User_Group/Past_Topics#The_Aggregate_Function|the Aggregate function]].

= Items within Budget function =

Author: Max Henrion

Given a set of items, with a priority and a cost for each, the function Items_within_budget function selects out the highest priority items that fit within the fixed budget. The function is available from: [[Media:Items_within_budget.ana|Items within budget]]

----
= Convolution =

Author: Lonnie Chrisman

The model [[Media:Convolution.ana|Convolution]] contains a function, Convolve(Y,Z,T,I), that computes the convolution of two time series.

A time series is a set of points, (Y,T), where T is the ascending X-axis, and the set of points is indexed by I. The values of T do not have to be equally spaced. The function treats Y and Z as being equal to 0 outside the range of T. The two time series here are the set of points (Y,T) and the set of points (Z,T), where both sets of points are indexed by I.

The model contains a couple examples of convolved functions.

The mathematical definition of the convolution of two time series is the function given by:

h(t) = Integral y(u) z(t-u) dt

Convolution is used predominantly in signal and systems analysis.

----
= Sampling from only feasible points =

Author: Lonnie Chrisman

Consider this scenario. You have a bunch of chance variables, each defined by a distribution. They joint sample generated, however, contains some combinations of points that are (for one reason or another) physically impossible. We'll call those infeasible points. You'd like to eliminate those points from the sample and keep only the feasible points.

The module [[Media:Feasible_Sampler.ana|Feasible Sampler]] implements a button that will sample a collection of chance variables, then reset the sample size and keep only those sample points that are "feasible".

Obviously, this approach will work best when most of your samples are feasible. If you can handle the "infeasible" points in your model directly, by conditioning certain chance variables on others, that is far preferable. But there are certainly cases where this solution (although a bit of a kludge) is more readily usable.

The instructions for how to use this are in the module description field.

----
= Grant Exclusion Model =

[[Media:Grant_exclusion.ANA|Grant Exclusion]]

This model tests a hypothesis about the distribution of an attribute of the marginal rejectee of a grant program, given the relevance of that attribute to award of the grant. It could be used by an organization to make decisions as to whether to fiscally-sponsor another organization that will use that fiscal sponsorship to apply for grants, by looking at the effect on the pool of grant recipients overall.

----
= Donor/Presenter Dashboard =

[[Media:Donor_Presenter_Dashboard_II.ANA|Donor-Presenter Dashboard]]

This model implements a continuous-time Markov chain in Analytica's discrete-time dynamic simulation environment. It supports immigration to, and emigration from, every node.

It can be used by an arts organization to probabilistically forecast future audience evolution, in both the short and the long (steady state) term. It also allows for uncertainty in the input parameters.

----

= Project Planner =

'''Download:''' [[Media:Project Priorities 5 0.ana|Project Priorities 5 0.ana]]

A demo model that shows how to:
* Evaluate a set of R&D projects, including uncertain R&D costs, and uncertain revenues if it leads to release of a commercial product.
* Use multiattribute analysis to compare projects, including a hard attribute -- expected net present value -- and soft attributes -- strategic fit, staff development, and public good will.
* Compare cost, NPV, and multiattribute value for a selected portfolio of projects.
* Generate the best portfolio (ratio of NPV or multiattribute merit to cost) given a R&D budget.

[[Image: Project planner model.png |500px]]

This link is only a test, and to an older version:
<link
target="blank">http://lumina.com/wiki/images/4/43/Project_priorities_2007_4.0.ANA</link>

----

= California Power Plants =

A model that demonstrates the use of [[Choice|choice pulldowns]] in edit tables. The model is created during a mini-tutorial on [[Inserting_Choice_Controls_in_Edit_Table_Cells]] elsewhere on this Wiki.

[[Media:California_Power_Plants.ANA]]

----

= Dependency Tracker Module =

This module tracks dependencies through your model, updating the visual appearance of nodes so that you can quickly visualize the paths by which one variable influences another. You can also use it to provide a visual indication of which nodes are downstream (or upstream) from an indicated variable.

The module contains button scripts that change the bevel appearance of nodes in your model. To see how Variable X influences Variable Y, the script will bevel the nodes for all variables that are influenced by X and influence Y. Alternatively, you can bevel all nodes that are influenced by X, or you can bevel all nodes that influence Y.

[[image:Dependency tracker.jpg]]

In the image above, the path from dp_ex_2 through dp_ex_4 has been highlight using the bevel style of the nodes. (The result of pressing the "Bevel all from Ancestor to Descendant" button)

[[media:Dependency_Tracker.ANA | Dependency_Tracker.ana]]

----

= Total Allowable Harvest =

The problem applies to any population of fish or animal whose dynamics are poorly known but can be summarized in a simple model:

N_t+1 = N_t*Lambda - landed catch*(1+loss rate)

where N_t is the population size (number of individuals) at time t, N_t+1 is the population size at time t+1, Lambda is the intrinsic rate of increase and the loss rate is the percentage of fish or animals killed but not retrieved relative to the landed catch, or catch secured.

The question here is to determine how many fish or animals can be caught (landed) annually so that the probability of the population declining X% in Y years (decline threshold) is less than Z% (risk tolerance).

Two models are available for download. One uses the Optimizer ([[NlpDefine]]) to find the maximum landed catch at the risk tolerance level for the given decline threshold. The other (for those using a version of Analytica without Optimizer) uses [[StepInterp]] in an iterative way to get that maximum landed catch.

* [[media:Total Allowable Removal model with Optimizer.ana | Total Allowable w Optimizer.ana]]
* [[media:Total Allowable Removal model w StepInterp.ana|Total Allowable w StepInterp.ana]]

Models contributed by Pierre Richard

----

= Earthquake Expenses =

An example of risk analysis with time-dependence and costs shifted over time.

Certain organizations (insurance companies, large companies, governments) incur expenses following earthquakes. This simplified demo model can be used to answer questions such as:
* What is the probability of more than one quake in a specific 10 year period.
* What is the probability that in my time window my costs exceed $X?

Assumptions in this model:
* Earthquakes are Poisson events with mean rate of once every 10 years.
* Damage caused by such quake is lognormally distributed, with mean $10M adn stddev of $6M.
* Cost of damage gets incurred over the period of a year from the date of the quake as equipment is replaced and buildings are repaired over time: 20% in 1st quarter after quake, 50% in 2nd quarter, 20% in 3rd quarter, 10% in 4th quarter.

Model file: [[media:Earthquake expenses.ana|Earthquake expenses.ana]]

----

= Regulation of Photosynthesis =

Author Lonnie Chrisman, Ph.D.

[[image:Photosynthesis fluorescence.jpg]]

A model of how photosynthesis is regulated inside a cyanobacteria. As light exposure varies over time (and you can experiment with various light intensity waveforms), it simulates the concentration levels of key transport molecules along the chain, through the PSII complex, plasto-quinone pool, PSI complex, down to metabolic oxidation. The dynamic response to light levels, or changes in light levels, over time becomes evident, and the impact of changes to metabolic demand can also be observed. In the graph of fluorescence above, we can see an indicator of how much energy is being absorbed, in three different cases (different light intensities). In the two higher intensity cases, photoinhibition is observed -- a protective mechanism of the cell that engages when more energy is coming in than can be utilized by the cell. Excess incoming energy, in the absense of photoinhibition, causes damage, particularly to the PSII complex.

This model uses node shapes for a different purpose than is normally seen in decision analysis models. In this model, ovals, instead of depicting chance variables, depict chemical reactions, where the value depicts the reaction rate, and rounded rectangles depict chemical concentrations.

Two models are attached. The first is a bit cleaner, and focused on the core transport chain, as described above. The second is less developed, but is focused more on genetic regulation processes.

* [[media:Photosynthesis regulation.ANA|Photosynthesis Regulation.ana]] - main regulation pathways
* [[media:Photosystem.ana | Photosystem.ana]] - rough sketch of genetic regulation.

----

= Cross-Validation / Fitting Kernel Functions to Data =

Author: Lonnie Chrisman, Ph.D., Lumina Decision Systems

[[image:Cross-validated data fit.jpg]]

Model: [[media:Cross-validation example.ana|Cross-validation example.ana]]

When fitting a function to data, if you have too many free parameters relative to the number of points in your data set, you may "overfit" the data. When this happens, the fit to your training data may be very good, but the fit to new data points (beyond those used for training) may be very poor.

Cross-validation is a common technique to deal with this problem. With this technique, we set aside a fraction of the available data as a cross-validation set. Then we begin by fitting very simple functions to the data (with very few free parameters), successively increasing the number of free parameters, and seeing how the predictive performance changes on the cross-validation set. It is typical to see improvement on the cross-validation set for a while, followed by a deterioriation of predictive performance on the cross-validation set once overfitting starts occuring.

This example model successively fits a non-linear kernel function to the residual error, and uses cross-validation to determine how many kernel functions should be used.

Requires Analytica Optimizer: The kernel fitting function (Kern_Fit) uses [[NlpDefine]].

----
= Statistical Bootstrapping =

Model: [[media:Bootstrapping.ana|Bootstrapping.ana]]

Bootstrapping is a technique from statistics for estimating the sampling error present in a statistical estimator. The simplest version estimates sampling error by resampling the original data. This model demonstrates how this is accomplished in Analytica.

----

= Compression Post Load Calculator =

Author: Lonnie Chrisman

Model: [[media:Compression_Post_Load_Capacity.ana|Compression_Post_Load_Capacity.ana]]

Computes the load that a Douglas-Fir Larch post can support in compression. Works for different timber types and grades and post sizes.

----
= Daylighting Options in Building Design =

Author: Max Henrion

Model: [[media:Daylighting analyzer.ana|Daylighting analyzer.ana]]

A demonstration showing how to analyze lifecycle costs and savings from daylighting options in building design.

Analysis based on Nomograph Cost/Benefit Tool for Daylighting. adapted from S.E. Selkowitz and M. Gabel. 1984. "LBL Daylighting Nomographs," LBL-13534, Lawrence Berkeley Laboratory, Berkeley CA, 94704. (510) 486-6845.

----
= Plane Catching Decision with EVIU =

Author: Max Henrion

Model: [[media:Plane catching decision with EVIU.ana|Plane catching decision with EVIU.ana]]

A simple model to assess what time I should leave my home to catch a plane, with uncertain driving time, walking from parking to gate (including security), and how long I need to be at the gate ahead of scheduled departure time. It uses a loss model based on minutes, assuming I value each extra minute snoozing in bed and set the loss if I miss the plane to 400 of those minutes.

It illustrates the EVIU (expected value of including uncertainty) i.e. the difference in expected value if I make a decision to minimize expected loss instead of decision to minimize time ignoring uncertainty (assuming each distribution is fixed at its mid value). For more details see "The Value of Knowing How Little You Know", Max Henrion, PhD Dissertation, Carnegie Mellon University, 1982.

----

= Marginal Analysis for Control of SO2 emissions =

Author: Surya Swamy

Model: [[media:Marginal Analysis for Control of SO2 Emissions.ana|Marginal Analysis for Control of SO2 Emissions.ana]]

Acid rain in eastern US and Canada caused by sulfur dioxide is emitted primarily by coal-burning electric-generating plants in the Midwestern U.S. This model demonstrates a marginal analysis a.k.a. benefit/cost analysis to determine the policy alternative that leads us to the most economically efficient level of cleanup.

----

= Electrical Generation and Transmission =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Electrical Transmission.ana|Electrical Transmission.ana]]

''Requires Analytica Optimizer''

This is a simple model of an electrical distribution network. At each node in the network we have power generators and power consumers (demand). Nodes are connected by branches (i.e., transmission lines), where each branch has a given admittance (the real part of impedeance is assumed to be zero) and a maximum capacity in Watts. Each power generator has a min and max generation capability with a given marginal rate per kilowatt-hour. The model uses a linear program to determine how much power each generator should produce so as to minimize total production cost, while satisfying demand and remaining within branch capacity constraints.

----
= Loan Policy Selection =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Loan policy selection.ANA|Loan policy selection.ANA]]

'''Best used with Analytica Optimizer'''

A lender has a large pool of money to loan, but needs to decide what credit rating threshold to require and what interest rate (above prime) to charge. The optimal value is determined by market forces (competiting lenders) and by the probability that the borrower defaults on the loan, which is a function of the economy and borrower's credit rating. The model can be used without the Analytica optimizer, in which case you can explore the decision space manually or use a parametric analysis to find the near optimal solution. Those with Analytica Optimizer can find the optimal solution (more quickly) using an [[NlpDefine|NLP]] search.
----

= Time-series re-indexing =

'''Author:''' Lonnie Chrisman

'''Model:''' [[media:Time-series-reindexing.ana|Time-series-reindexing.ana]]

This model contains some examples of time-series re-indexing. It is intended to demonstrate some of these basic techniques.

[[image:Weekly_data_graph_ex.png]]

In this example, actual measurements were collected at non-uniform time increments. Before analyzing these, we map these to a uniformly spaced time index (Week), occuring on Monday of each week. The mapping is done using an interpolation. The evenly-spaced data is then used to forecast future behavior. We first forecast over an index containing only future time points (Future_weeks), using a log-normal process model based on the historical weekly change. We then combine the historical data with the forecast on a common index (Week). A prob-bands graph of the weekly_data result shows the range of uncertainty projected by the process model (you'll notice the uncertainty exists only for future forecasted values, not historical ones).

----

= Multi-lingual Influence Diagram =

'''Author:''' D. Rice, Lumina Decision Systems.

'''Model:''' [[media:French-English.ana|French-English.ana]]

{| border="0"
| [[Image:English-view.png]]
| [[Image:French-view.png]]
|}

Maintains a single influence diagram with Title and Description attributes in both English and French. With the change of a pull-down, the influence diagram and all object descriptions are instantly reflected in the language of choice.

If you change a title or description while viewing English, and then change to French, the change you made will become the English-language version of the description. Similarly if you make a change while viewing French.

= Smooth PDF plots using Kernel Density Estimation =

'''Author:''' D. Rice, Lumina Decision Systems

'''Model:''' [[media:Kernel_Density_Estimation.ana|Kernel_Density_Estimation.ana]]

{| border="0"
| [[image:Dens_Est_builtin_pdf.png|frame|Analytica's built-in PDF plot with default settings]]
|
[[image:Dens_Est_Kernel_pdf.png|frame|PDF computed from Kernel Density estimation]]
|}

This example demonstrates a very simple fixed-width kernel density estimator to estimate a "smooth" probability density. The built-in PDF function in Analytica often has a choppy appearance due to the nature of histogramming -- it sets up a set of bins and counts how many points land in each bin. A kernel density estimator smooths this out, producing a less choppy PDF plot.

= Output and Input Columns in Same Table =

'''Author:''' D. Rice, Lumina Decision Systems

'''Model:''' [[media:Output and input columns.ana|Output and input columns.ana]]

[[image:Output and input columns.png]]

Presents an input table to a user, where one column is populated with computed output data, the other column with checkboxes for the user to select. Although the '''Output Data''' column isn't read only, as would be desired, a [[Check Attribute]] has been configured to complain if he does try to change values in that column. The model that uses these inputs would ignore any changes he makes to data in the '''Output Data''' column.

Populating the '''Output Data''' column requires the user to press a button, which runs a button script to populate that column. This button is presented on the top-level panel. If you change the input value, the output data will change, and then the button needs to be pressed to refresh the output data column.

= Linearizing a discrete NSP =

'''Author:''' P. Sanford, Lumina Decision Systems

'''Model:''' [[media:Output and input columns.ana|Output and input columns.ana]]

Cereal formulation model

File:Optimal Box.ana

2011-03-24T19:28:46Z

Psanford: Simple NLP demonstrated in interactive working session

Simple NLP demonstrated in interactive working session

File:Call center.ana

2011-03-24T19:21:44Z

Psanford: An LP developed and debugged during the interactive webinar. Also includes a challenge problem not presented in the webinar

An LP developed and debugged during the interactive webinar. Also includes a challenge problem not presented in the webinar

Analytica User Group

2011-03-15T22:24:56Z

Psanford: /* Interactive Optimization Workshop */

[[/Past Topics|Past Topics]] >>

The Analytica User Group is a way for Analytica users to support each other by sharing tips and function libraries. It includes a webinar series with tutorials and demonstrations on key Analytica features.

= Webinar Series =

Analytica User Webinars are a great way to learn about key Analytica features and modeling skills. They are live demos, using Citrix Gotowebinar, and VOIP or your telephone, at your choice. Questions, comments, and tangents are welcome.

There are webinars most weeks, usually on Thursdays at 10am PST (1pm EST). Seats are limited. To sign up for a particular webinar, see "How to Attend" below. Presentations may last anywhere from 20 to 90 minutes (with an estimate provided upfront).

If you missed a User Webinar, don't despair. We usually records them, including audio and screensharing. So, you can replay them any time. See below for the full list.

;Recent talks include:
* [[/Past_Topics#Introduction_to_Structured_Optimization|Introduction to Structured Optimization]] (24 Feb 2011)
* [[/Past_Topics#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]]
** [[/Past_Topics#Session_1:_Uncertainty_and_Probability|Session 1]]: (29 Apr 2010)[[/Past_Topics#Session_1:_Uncertainty_and_Probability|Introduction to Uncertainty, Probability]].
** [[/Past_Topics#Session_2:_Probability_Distributions|Session 2]]: (6 May 2010)[[/Past_Topics#Session_2:_Probability_Distributions|Probabiliity Distributions]]
** [[/Past_Topics#Session_3:_Monte_Carlo|Session 3]]: (13 May 2010)[[/Past_Topics#Session_3:_Monte_Carlo|Monte Carlo Simulation]]
** [[/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Session 4]]: (20 May 2010) [/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Measures of Risk and Utility]]
** [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Session 5]]: (3 June 2010) [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Risk Analysis for Portfolios]]
** [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Session 6]]: (10 June 2010) [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Common Parametric Distributions]]
** [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Session 7]]: (24 June 2010) [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Expert Assessment of Uncertainty]]
** [[/Past_Topics#Session_8:_Hypothesis_Testing|Session 8]]: (15 July 2010) [[/Past_Topics#Session_8:_Hypothesis Testing|Statistical Hypothesis Testing]]
* [[/Past_Topics#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[/Past_Topics#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[/Past Topics#Spearman_Rank_Correlation|Introduction to Spearman's Rank Correlation]] (25 Mar 2010)

== Schedule of Upcoming Webinars ==

=== Interactive Optimization Workshop ===

'''Date and Time:''' Thursday, 24 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Paul Sanford, Lumina Decision Systems

'''Abstract'''

This is an interactive workshop where you will learn the basics of creating Structured Optimization models and challenge yourself to set up and solve some basic examples on your own! No prior training in optimization is required. '''Trial Downloads'''[http://www.lumina.com/products/optimizer-trial/] of Analytica Optimizer 4.3 are now available. Attendees are encouraged to have Analytica Optimizer 4.3 installed and running during the workshop.

=== Optimizing Parameters in a Complex Model to Match Historical Data ===

'''Date and Time:''' Thursday, 31 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Almost all quantitative models have parameters that must be assessed by experts or estimated from historical data. Estimation from historical data can be complicated by the presence of variables that are either unobservable or unavailable in the historical record. Maximum likelihood estimation addresses this by finding the parameter settings that maximize the likelihood of the historical data predicted by the model. In this talk, I will formulate the parameter fitting task as a structured optimization problem (NLP), providing a hands-on demonstration of the new structured optimization features in Analytica 4.3.

=== Optimization with Uncertainty ===

'''Date and Time:''' Thursday, 6 April 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Analytica analyzes uncertainty by conducting a Monte Carlo analysis. When you optimize decision variables in a model containing uncertainty, you have a choice: You can perform one optimization over the Monte Carlo analysis, or you can perform a Monte Carlo sampling of optimizations (i.e., the Monte Carlo is inside the optimization, or the optimization is inside the Monte Carlo). The first case is used when the decision must be taken while the quantities are still uncertain. The second case is used when the values of the uncertain quantities will be resolved before the decisions are taken.

To illustrate, consider the situation faced by a relief organization that provides aid to victims of large natural disasters. In one situation, a decision must be made regarding how many resources to deploy to one particular location that has been hit by a large tsunami. At the time the decision must be made, the number of casualties is highly uncertain. In a different situation, the organization wants to characterize the uncertainty in its need for resources, given that it will optimally deploy resources in response to natural disasters as they occur.

=== Neural Networks ===

'''Date and Time:''' Thursday, 13 April 2011, 10:00am Pacific Daylight Time

=== Linearizing Optimization Models ===

=== [[Model building by mouse]] ===

'''Date and Time:''' (TBD)

'''Presenter''': (TBD)

'''Abstract'''

See Model building by mouse for the concept. The talk will also cover the TemplateInput and TemplateOutput attributes first introduced in 4.2 that make template modules possible, and thus expand the potential power and applicability of this modeling technique.

=== Saving Memory: Controlling when results are (and are not) cached ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

When a variable is computed, Analytica caches the result (i.e., keeps a copy around) so that when the same result is needed later, it does not need to be recomputed. Large array-valued results can end up consuming large amounts of memory, and lead to annoyances such as running out of available memory. There are instances where it isn't really necessary to cache a result -- either it will never be needed a second time, or it isn't a big deal to recompute it, or it is an exact copy of its parent (perhaps because only the result graph settings are different), so that by configuring that variable to not cache its result, or to release its cache as soon as all its children are computed, memory space can be recovered, perhaps allowing larger simulations to be conducted.

Analytica 4.2 introduces a new feature allowing you to configure when results are (or are not) cached. I'll demonstrate how to utilize this feature, and discuss limitations, such as situations where it would be a very bad idea not to cache.

If time allows (it probably will), I'll also throw in two other small memory-related controls new in 4.2. The [[WhatIf]]-style functions ([[WhatIf]], [[WhatIfAll]], [[Dydx]], [[Elasticity]], [[NlpDefine]]), i.e., functions that compute hypotheticals, have been modified in 4.2 to preserve previously computed values of other variables. This is a nice feature, but can impact tight-memory cases. I'll explain this change and how to avoid the extra memory consumption where appropriate. Second, I'll also show how Analytica's maximum working set size can be configured (on some operating systems), which can be used to keep other applications on your computer highly responsive even when large memory-intensive computations are churning away.

In Analytica 64-bit, you are less likely to encounter the problem of running out of memory, but you still may encounter performance slowdown from huge memory utilization. Some large models, requiring extensive memory resources, run smoothly even when requiring far more memory than there is RAM available, while other result in extensive ''thrashing'', taking very long to evaluate. I've found that differentiating factor has to do with the size of individual arrays within the model. Thrashing tends to occur when a single array consumes more than about 1/3 the available RAM, while if all arrays in the model remain small relative to available RAM, huge memory evaluations often run very smoothly. I'll relate these observations, and what that means with respect to sample size.

This talk is appropriate for advanced Analytica modelers.

=== The [[Dynamic]] Function and Dynamic Loops ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function in Analytica 4.2 is a bit more flexible than previously. It is now possible for [[User-Defined Functions]] to be part of a dynamic loop, and there are cases where it is now possible to operate over the [[Time]] index where previous it was not. I'll discuss how dynamic models are evaluated, and how that differs from non-dynamic models, and cover some tricks for tracing evaluation and debugging complex dynamic models (which can often be challenging to debug). I'll also introduce how [[Dynamic]] can be used on an index other than the built-in [[Time]] index.

=== [[IntraTable]]s ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

An [[IntraTable]] is a variation of an edit [[Table]], where (like [[Table]]s), the cells may contain expressions, but unlike [[Table]]s, the expressions in each cell can refer to values of other cells. This may (quite appropriately) conjure up nightmares of spreadsheet hell. While this isn't something you'd want to use nonchalantly, there are cases when the ability to reference other cells within the same table comes with its benefits. Because the [[IntraTable]] allows any pattern of references (it doesn't have to be strictly left-to-right, for example), as long as a cell-cycle isn't created, it is possible to encode certain recurrences that would otherwise be quite difficult to encode.

== How to Attend ==

To attend, you need to sign up by contacting Lumina at mailto:webinars@lumina.com. Please sign up at least a day prior. Attendance is limited, so please don't sign up unless you sincerely intend to attend.

These Webinars are FREE to users who have an up-to-date Support for Analytica. If you are unsure, check with mailto:sales@lumina. For those without current support, the fee is US$50.

== How to be a Presenter ==

Being a presenter at an Analytica webinar provides an opportunity to make others in the Analytica community aware of your successes or capabilities. Consultants may find this an opportunity for exposure to others with particular modeling needs. Also, if you are an Analytica aficionado, this is a great opportunity to help others.

If you would like to be a presenter, submit your proposed topic to webinars@lumina.com and possible presentation times (include the time zone). We will schedule the GotoMeeting conference (you do not need a gotoMeeting subscription yourself) and we will make you presenter during the session, allowing you to share your screen while you talk. You will most likely make use of Power Point and a running Analytica during your presentation.

==Analytica User Group:([[Analytica_User_Group/Past_Topics#The_Large_Sample_Library|Large Sample Library]])==

== [[Analytica User Group/Past Topics|Archive of Past Topics]] ==

Visit the [[Analytica User Group/Past Topics|Archive of Past Topics]] for abstracts, example models, and recordings of previous Analytica User Group webinars. Previous topics that have been presented include:

* Guidelines for Model Transparency
* Automated Monitoring and Failure Detection
* Principal Component Analysis (PCA)
* Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]])
* Bond Portfolio Analysis
* Net Present Value ([[NPV]])
* The Analytica Wiki, and How to Contribute
* Data Center Capacity Planning
* Modeling the Precision Strike Process
* The Performance Profiler
* Variable Stiffness Cubic Splines
* Introduction to [[DetermTable]]s
* Importance Sampling (Bayesian posteriors)
* Importance Sampling (rare events)
* Statistical Functions
* [[Table Splicing]]
* Analytica Web Player
* [[SubTable|SubTables]]
* Creating Custom Distribution Functions
* Using the Check attribute to validate inputs and results
* The [[Dynamic]] Function
* [[Logistic Regression]]
* Creating Control Panels
* Statistical Functions
* Creating Scatter Plots
* Using [[Regression]]
* New Functions for Reading Directly from an Excel File
* Advanced Tornado Charts -- when inputs are Array-Valued
* Using the Analytica Decision Engine (ADE) from ASP.NET
* OLE Linking
* Tornado Charts
* Correlated and Multivariate Distributions
* Assessment of Probability Distributions
* Graph Style Templates
* Sneak preview of Analytica Web Publisher
* Querying an OLAP server
* Querying an ODBC relational database
* [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]]
* Flattening and Unflattening of Arrays
* Introduction to Arrays and Array Abstraction
* [[Local Indexes]]
* The [[Iterate]] Function
* The [[Using_References|Reference and Dereference Operators]]
* Modeling Utility Tariffs in Analytica
* Modeling Energy Efficiency in Large Data Centers
* Calling External Applications
* Introduction to Linear and Quadratic Programming
* Non-Linear Optimization
* Writing [[User-Defined Functions]]
* Modeling Markov Processes in Analytica
* Manipulating Dates in Analytica
* Button Scripting
* Manipulating Indexes and Arrays in Analytica Expressions
* Edit Table Enhancements in Analytica 4.0
* [[Handle]]s and [[Meta-Inference]]

== Potential future topics ==

If you would like to see a webinar on a given topic, please feel free to add it here. If you see a topic listed and would like to be a presenter, let us know.

* Statistical hypothesis testing.
** Standard textbook tests
** Computing p-values using Monte Carlo for complex and non-standard statistical models.

* Time-series analysis

* Using the interpolation functions, [[LinearInterp]], [[CubicInterp]], [[StepInterp]].

* Numeric precision - numeric round-off, underflow, etc., why they happen, what to do about it.

* Net Present Value and Internal Rate of Return -- introduction to the use of these commonly used metrics for quantifying decision quality.

* Using the ''Performance Profiler'' to understand where your model consumes time and memory resources.

* Producing graphs from ADE: Including how to serve graphs from web pages.

* Intracacies of the Domain attribute

* Mastering Array Abstraction

* Writing User-Defined Distribution functions.

* Getting data into Analytica.

* [[DetermTable]]s

* Large-scale sampling. Techniques when memory limitations constrain sampleSize.

* Creating User-Defined functions. Review of [[Function Parameter Qualifiers]].

* Understanding [[Evaluation Contexts]]

* Bayesian Inference

* Sensitivity Analysis.

* Dynamic Programming

* Approximate and Stochastic Dynamic Programming

== User Survey Results ==

During the first week of September, we sent out a survey to people who had attended webinars so far. Please continue providing us with feedback. Here is some feedback to date:

Level of difficult and speed:

So far, of those who answered this question, 25% say "a bit too easy", 25% say "too hard/fast", and 50% say "just right".

Topics requested for future webinars:

* How to run multiple iterations, e.g. 100 iterations with uncertainty sample of 1000.

* Sampling for rare events.

* Using the lognormal function.

* Financial modeling

* New array functionality (subtables, choices in tables)

* Optimizer !!!

* Dynamic models (twice requested)

* Choice of distributions

* Re-sampling and radomize methods and uncertainty sample size

* Tricks for sensitivity analysis

* Input and output nodes

* Importance analysis

= The Analytica Wiki =

The [[Analytica Wiki ]] contains many resources, including in-depth reference materials, relevant articles, example models, tutorials, etc., to help users master Analytica and find what they need. Even better, Analytica users can contribute! You can upload your own models, articles, expand on or correct materials that are there, etc., for the benefit of the entire Analytica community.

Analytica User Group

2011-03-15T22:24:27Z

Psanford: /* Interactive Optimization Workshop */

[[/Past Topics|Past Topics]] >>

The Analytica User Group is a way for Analytica users to support each other by sharing tips and function libraries. It includes a webinar series with tutorials and demonstrations on key Analytica features.

= Webinar Series =

Analytica User Webinars are a great way to learn about key Analytica features and modeling skills. They are live demos, using Citrix Gotowebinar, and VOIP or your telephone, at your choice. Questions, comments, and tangents are welcome.

There are webinars most weeks, usually on Thursdays at 10am PST (1pm EST). Seats are limited. To sign up for a particular webinar, see "How to Attend" below. Presentations may last anywhere from 20 to 90 minutes (with an estimate provided upfront).

If you missed a User Webinar, don't despair. We usually records them, including audio and screensharing. So, you can replay them any time. See below for the full list.

;Recent talks include:
* [[/Past_Topics#Introduction_to_Structured_Optimization|Introduction to Structured Optimization]] (24 Feb 2011)
* [[/Past_Topics#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]]
** [[/Past_Topics#Session_1:_Uncertainty_and_Probability|Session 1]]: (29 Apr 2010)[[/Past_Topics#Session_1:_Uncertainty_and_Probability|Introduction to Uncertainty, Probability]].
** [[/Past_Topics#Session_2:_Probability_Distributions|Session 2]]: (6 May 2010)[[/Past_Topics#Session_2:_Probability_Distributions|Probabiliity Distributions]]
** [[/Past_Topics#Session_3:_Monte_Carlo|Session 3]]: (13 May 2010)[[/Past_Topics#Session_3:_Monte_Carlo|Monte Carlo Simulation]]
** [[/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Session 4]]: (20 May 2010) [/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Measures of Risk and Utility]]
** [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Session 5]]: (3 June 2010) [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Risk Analysis for Portfolios]]
** [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Session 6]]: (10 June 2010) [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Common Parametric Distributions]]
** [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Session 7]]: (24 June 2010) [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Expert Assessment of Uncertainty]]
** [[/Past_Topics#Session_8:_Hypothesis_Testing|Session 8]]: (15 July 2010) [[/Past_Topics#Session_8:_Hypothesis Testing|Statistical Hypothesis Testing]]
* [[/Past_Topics#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[/Past_Topics#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[/Past Topics#Spearman_Rank_Correlation|Introduction to Spearman's Rank Correlation]] (25 Mar 2010)

== Schedule of Upcoming Webinars ==

=== Interactive Optimization Workshop ===

'''Date and Time:''' Thursday, 24 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Paul Sanford, Lumina Decision Systems

'''Abstract'''
This is an interactive workshop where you will learn the basics of creating Structured Optimization models and challenge yourself to set up and solve some basic examples on your own! No prior training in optimization is required. '''Trial Downloads'''[http://www.lumina.com/products/optimizer-trial/] of Analytica Optimizer 4.3 are now available. Attendees are encouraged to have Analytica Optimizer 4.3 installed and running during the workshop.

=== Optimizing Parameters in a Complex Model to Match Historical Data ===

'''Date and Time:''' Thursday, 31 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Almost all quantitative models have parameters that must be assessed by experts or estimated from historical data. Estimation from historical data can be complicated by the presence of variables that are either unobservable or unavailable in the historical record. Maximum likelihood estimation addresses this by finding the parameter settings that maximize the likelihood of the historical data predicted by the model. In this talk, I will formulate the parameter fitting task as a structured optimization problem (NLP), providing a hands-on demonstration of the new structured optimization features in Analytica 4.3.

=== Optimization with Uncertainty ===

'''Date and Time:''' Thursday, 6 April 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Analytica analyzes uncertainty by conducting a Monte Carlo analysis. When you optimize decision variables in a model containing uncertainty, you have a choice: You can perform one optimization over the Monte Carlo analysis, or you can perform a Monte Carlo sampling of optimizations (i.e., the Monte Carlo is inside the optimization, or the optimization is inside the Monte Carlo). The first case is used when the decision must be taken while the quantities are still uncertain. The second case is used when the values of the uncertain quantities will be resolved before the decisions are taken.

To illustrate, consider the situation faced by a relief organization that provides aid to victims of large natural disasters. In one situation, a decision must be made regarding how many resources to deploy to one particular location that has been hit by a large tsunami. At the time the decision must be made, the number of casualties is highly uncertain. In a different situation, the organization wants to characterize the uncertainty in its need for resources, given that it will optimally deploy resources in response to natural disasters as they occur.

=== Neural Networks ===

'''Date and Time:''' Thursday, 13 April 2011, 10:00am Pacific Daylight Time

=== Linearizing Optimization Models ===

=== [[Model building by mouse]] ===

'''Date and Time:''' (TBD)

'''Presenter''': (TBD)

'''Abstract'''

See Model building by mouse for the concept. The talk will also cover the TemplateInput and TemplateOutput attributes first introduced in 4.2 that make template modules possible, and thus expand the potential power and applicability of this modeling technique.

=== Saving Memory: Controlling when results are (and are not) cached ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

When a variable is computed, Analytica caches the result (i.e., keeps a copy around) so that when the same result is needed later, it does not need to be recomputed. Large array-valued results can end up consuming large amounts of memory, and lead to annoyances such as running out of available memory. There are instances where it isn't really necessary to cache a result -- either it will never be needed a second time, or it isn't a big deal to recompute it, or it is an exact copy of its parent (perhaps because only the result graph settings are different), so that by configuring that variable to not cache its result, or to release its cache as soon as all its children are computed, memory space can be recovered, perhaps allowing larger simulations to be conducted.

Analytica 4.2 introduces a new feature allowing you to configure when results are (or are not) cached. I'll demonstrate how to utilize this feature, and discuss limitations, such as situations where it would be a very bad idea not to cache.

If time allows (it probably will), I'll also throw in two other small memory-related controls new in 4.2. The [[WhatIf]]-style functions ([[WhatIf]], [[WhatIfAll]], [[Dydx]], [[Elasticity]], [[NlpDefine]]), i.e., functions that compute hypotheticals, have been modified in 4.2 to preserve previously computed values of other variables. This is a nice feature, but can impact tight-memory cases. I'll explain this change and how to avoid the extra memory consumption where appropriate. Second, I'll also show how Analytica's maximum working set size can be configured (on some operating systems), which can be used to keep other applications on your computer highly responsive even when large memory-intensive computations are churning away.

In Analytica 64-bit, you are less likely to encounter the problem of running out of memory, but you still may encounter performance slowdown from huge memory utilization. Some large models, requiring extensive memory resources, run smoothly even when requiring far more memory than there is RAM available, while other result in extensive ''thrashing'', taking very long to evaluate. I've found that differentiating factor has to do with the size of individual arrays within the model. Thrashing tends to occur when a single array consumes more than about 1/3 the available RAM, while if all arrays in the model remain small relative to available RAM, huge memory evaluations often run very smoothly. I'll relate these observations, and what that means with respect to sample size.

This talk is appropriate for advanced Analytica modelers.

=== The [[Dynamic]] Function and Dynamic Loops ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function in Analytica 4.2 is a bit more flexible than previously. It is now possible for [[User-Defined Functions]] to be part of a dynamic loop, and there are cases where it is now possible to operate over the [[Time]] index where previous it was not. I'll discuss how dynamic models are evaluated, and how that differs from non-dynamic models, and cover some tricks for tracing evaluation and debugging complex dynamic models (which can often be challenging to debug). I'll also introduce how [[Dynamic]] can be used on an index other than the built-in [[Time]] index.

=== [[IntraTable]]s ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

An [[IntraTable]] is a variation of an edit [[Table]], where (like [[Table]]s), the cells may contain expressions, but unlike [[Table]]s, the expressions in each cell can refer to values of other cells. This may (quite appropriately) conjure up nightmares of spreadsheet hell. While this isn't something you'd want to use nonchalantly, there are cases when the ability to reference other cells within the same table comes with its benefits. Because the [[IntraTable]] allows any pattern of references (it doesn't have to be strictly left-to-right, for example), as long as a cell-cycle isn't created, it is possible to encode certain recurrences that would otherwise be quite difficult to encode.

== How to Attend ==

To attend, you need to sign up by contacting Lumina at mailto:webinars@lumina.com. Please sign up at least a day prior. Attendance is limited, so please don't sign up unless you sincerely intend to attend.

These Webinars are FREE to users who have an up-to-date Support for Analytica. If you are unsure, check with mailto:sales@lumina. For those without current support, the fee is US$50.

== How to be a Presenter ==

Being a presenter at an Analytica webinar provides an opportunity to make others in the Analytica community aware of your successes or capabilities. Consultants may find this an opportunity for exposure to others with particular modeling needs. Also, if you are an Analytica aficionado, this is a great opportunity to help others.

If you would like to be a presenter, submit your proposed topic to webinars@lumina.com and possible presentation times (include the time zone). We will schedule the GotoMeeting conference (you do not need a gotoMeeting subscription yourself) and we will make you presenter during the session, allowing you to share your screen while you talk. You will most likely make use of Power Point and a running Analytica during your presentation.

==Analytica User Group:([[Analytica_User_Group/Past_Topics#The_Large_Sample_Library|Large Sample Library]])==

== [[Analytica User Group/Past Topics|Archive of Past Topics]] ==

Visit the [[Analytica User Group/Past Topics|Archive of Past Topics]] for abstracts, example models, and recordings of previous Analytica User Group webinars. Previous topics that have been presented include:

* Guidelines for Model Transparency
* Automated Monitoring and Failure Detection
* Principal Component Analysis (PCA)
* Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]])
* Bond Portfolio Analysis
* Net Present Value ([[NPV]])
* The Analytica Wiki, and How to Contribute
* Data Center Capacity Planning
* Modeling the Precision Strike Process
* The Performance Profiler
* Variable Stiffness Cubic Splines
* Introduction to [[DetermTable]]s
* Importance Sampling (Bayesian posteriors)
* Importance Sampling (rare events)
* Statistical Functions
* [[Table Splicing]]
* Analytica Web Player
* [[SubTable|SubTables]]
* Creating Custom Distribution Functions
* Using the Check attribute to validate inputs and results
* The [[Dynamic]] Function
* [[Logistic Regression]]
* Creating Control Panels
* Statistical Functions
* Creating Scatter Plots
* Using [[Regression]]
* New Functions for Reading Directly from an Excel File
* Advanced Tornado Charts -- when inputs are Array-Valued
* Using the Analytica Decision Engine (ADE) from ASP.NET
* OLE Linking
* Tornado Charts
* Correlated and Multivariate Distributions
* Assessment of Probability Distributions
* Graph Style Templates
* Sneak preview of Analytica Web Publisher
* Querying an OLAP server
* Querying an ODBC relational database
* [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]]
* Flattening and Unflattening of Arrays
* Introduction to Arrays and Array Abstraction
* [[Local Indexes]]
* The [[Iterate]] Function
* The [[Using_References|Reference and Dereference Operators]]
* Modeling Utility Tariffs in Analytica
* Modeling Energy Efficiency in Large Data Centers
* Calling External Applications
* Introduction to Linear and Quadratic Programming
* Non-Linear Optimization
* Writing [[User-Defined Functions]]
* Modeling Markov Processes in Analytica
* Manipulating Dates in Analytica
* Button Scripting
* Manipulating Indexes and Arrays in Analytica Expressions
* Edit Table Enhancements in Analytica 4.0
* [[Handle]]s and [[Meta-Inference]]

== Potential future topics ==

If you would like to see a webinar on a given topic, please feel free to add it here. If you see a topic listed and would like to be a presenter, let us know.

* Statistical hypothesis testing.
** Standard textbook tests
** Computing p-values using Monte Carlo for complex and non-standard statistical models.

* Time-series analysis

* Using the interpolation functions, [[LinearInterp]], [[CubicInterp]], [[StepInterp]].

* Numeric precision - numeric round-off, underflow, etc., why they happen, what to do about it.

* Net Present Value and Internal Rate of Return -- introduction to the use of these commonly used metrics for quantifying decision quality.

* Using the ''Performance Profiler'' to understand where your model consumes time and memory resources.

* Producing graphs from ADE: Including how to serve graphs from web pages.

* Intracacies of the Domain attribute

* Mastering Array Abstraction

* Writing User-Defined Distribution functions.

* Getting data into Analytica.

* [[DetermTable]]s

* Large-scale sampling. Techniques when memory limitations constrain sampleSize.

* Creating User-Defined functions. Review of [[Function Parameter Qualifiers]].

* Understanding [[Evaluation Contexts]]

* Bayesian Inference

* Sensitivity Analysis.

* Dynamic Programming

* Approximate and Stochastic Dynamic Programming

== User Survey Results ==

During the first week of September, we sent out a survey to people who had attended webinars so far. Please continue providing us with feedback. Here is some feedback to date:

Level of difficult and speed:

So far, of those who answered this question, 25% say "a bit too easy", 25% say "too hard/fast", and 50% say "just right".

Topics requested for future webinars:

* How to run multiple iterations, e.g. 100 iterations with uncertainty sample of 1000.

* Sampling for rare events.

* Using the lognormal function.

* Financial modeling

* New array functionality (subtables, choices in tables)

* Optimizer !!!

* Dynamic models (twice requested)

* Choice of distributions

* Re-sampling and radomize methods and uncertainty sample size

* Tricks for sensitivity analysis

* Input and output nodes

* Importance analysis

= The Analytica Wiki =

The [[Analytica Wiki ]] contains many resources, including in-depth reference materials, relevant articles, example models, tutorials, etc., to help users master Analytica and find what they need. Even better, Analytica users can contribute! You can upload your own models, articles, expand on or correct materials that are there, etc., for the benefit of the entire Analytica community.

Analytica User Group

2011-03-15T22:21:46Z

Psanford: /* Interactive Optimization Workshop */

[[/Past Topics|Past Topics]] >>

The Analytica User Group is a way for Analytica users to support each other by sharing tips and function libraries. It includes a webinar series with tutorials and demonstrations on key Analytica features.

= Webinar Series =

Analytica User Webinars are a great way to learn about key Analytica features and modeling skills. They are live demos, using Citrix Gotowebinar, and VOIP or your telephone, at your choice. Questions, comments, and tangents are welcome.

There are webinars most weeks, usually on Thursdays at 10am PST (1pm EST). Seats are limited. To sign up for a particular webinar, see "How to Attend" below. Presentations may last anywhere from 20 to 90 minutes (with an estimate provided upfront).

If you missed a User Webinar, don't despair. We usually records them, including audio and screensharing. So, you can replay them any time. See below for the full list.

;Recent talks include:
* [[/Past_Topics#Introduction_to_Structured_Optimization|Introduction to Structured Optimization]] (24 Feb 2011)
* [[/Past_Topics#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]]
** [[/Past_Topics#Session_1:_Uncertainty_and_Probability|Session 1]]: (29 Apr 2010)[[/Past_Topics#Session_1:_Uncertainty_and_Probability|Introduction to Uncertainty, Probability]].
** [[/Past_Topics#Session_2:_Probability_Distributions|Session 2]]: (6 May 2010)[[/Past_Topics#Session_2:_Probability_Distributions|Probabiliity Distributions]]
** [[/Past_Topics#Session_3:_Monte_Carlo|Session 3]]: (13 May 2010)[[/Past_Topics#Session_3:_Monte_Carlo|Monte Carlo Simulation]]
** [[/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Session 4]]: (20 May 2010) [/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Measures of Risk and Utility]]
** [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Session 5]]: (3 June 2010) [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Risk Analysis for Portfolios]]
** [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Session 6]]: (10 June 2010) [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Common Parametric Distributions]]
** [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Session 7]]: (24 June 2010) [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Expert Assessment of Uncertainty]]
** [[/Past_Topics#Session_8:_Hypothesis_Testing|Session 8]]: (15 July 2010) [[/Past_Topics#Session_8:_Hypothesis Testing|Statistical Hypothesis Testing]]
* [[/Past_Topics#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[/Past_Topics#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[/Past Topics#Spearman_Rank_Correlation|Introduction to Spearman's Rank Correlation]] (25 Mar 2010)

== Schedule of Upcoming Webinars ==

=== Interactive Optimization Workshop ===

'''Date and Time:''' Thursday, 24 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Paul Sanford, Lumina Decision Systems

'''Abstract'''
This is an interactive workshop where you will learn the basics of creating Structured Optimization models and challenge yourself to set up and solve some basic examples on your own! No prior training in optimization is required. [http://www.lumina.com/products/optimizer-trial/]Trial Downloads of Analytica Optimizer 4.3 are now available. Attendees are encouraged to have Analytica Optimizer 4.3 installed and running during the workshop.

=== Optimizing Parameters in a Complex Model to Match Historical Data ===

'''Date and Time:''' Thursday, 31 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Almost all quantitative models have parameters that must be assessed by experts or estimated from historical data. Estimation from historical data can be complicated by the presence of variables that are either unobservable or unavailable in the historical record. Maximum likelihood estimation addresses this by finding the parameter settings that maximize the likelihood of the historical data predicted by the model. In this talk, I will formulate the parameter fitting task as a structured optimization problem (NLP), providing a hands-on demonstration of the new structured optimization features in Analytica 4.3.

=== Optimization with Uncertainty ===

'''Date and Time:''' Thursday, 6 April 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Analytica analyzes uncertainty by conducting a Monte Carlo analysis. When you optimize decision variables in a model containing uncertainty, you have a choice: You can perform one optimization over the Monte Carlo analysis, or you can perform a Monte Carlo sampling of optimizations (i.e., the Monte Carlo is inside the optimization, or the optimization is inside the Monte Carlo). The first case is used when the decision must be taken while the quantities are still uncertain. The second case is used when the values of the uncertain quantities will be resolved before the decisions are taken.

To illustrate, consider the situation faced by a relief organization that provides aid to victims of large natural disasters. In one situation, a decision must be made regarding how many resources to deploy to one particular location that has been hit by a large tsunami. At the time the decision must be made, the number of casualties is highly uncertain. In a different situation, the organization wants to characterize the uncertainty in its need for resources, given that it will optimally deploy resources in response to natural disasters as they occur.

=== Neural Networks ===

'''Date and Time:''' Thursday, 13 April 2011, 10:00am Pacific Daylight Time

=== Linearizing Optimization Models ===

=== [[Model building by mouse]] ===

'''Date and Time:''' (TBD)

'''Presenter''': (TBD)

'''Abstract'''

See Model building by mouse for the concept. The talk will also cover the TemplateInput and TemplateOutput attributes first introduced in 4.2 that make template modules possible, and thus expand the potential power and applicability of this modeling technique.

=== Saving Memory: Controlling when results are (and are not) cached ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

When a variable is computed, Analytica caches the result (i.e., keeps a copy around) so that when the same result is needed later, it does not need to be recomputed. Large array-valued results can end up consuming large amounts of memory, and lead to annoyances such as running out of available memory. There are instances where it isn't really necessary to cache a result -- either it will never be needed a second time, or it isn't a big deal to recompute it, or it is an exact copy of its parent (perhaps because only the result graph settings are different), so that by configuring that variable to not cache its result, or to release its cache as soon as all its children are computed, memory space can be recovered, perhaps allowing larger simulations to be conducted.

Analytica 4.2 introduces a new feature allowing you to configure when results are (or are not) cached. I'll demonstrate how to utilize this feature, and discuss limitations, such as situations where it would be a very bad idea not to cache.

If time allows (it probably will), I'll also throw in two other small memory-related controls new in 4.2. The [[WhatIf]]-style functions ([[WhatIf]], [[WhatIfAll]], [[Dydx]], [[Elasticity]], [[NlpDefine]]), i.e., functions that compute hypotheticals, have been modified in 4.2 to preserve previously computed values of other variables. This is a nice feature, but can impact tight-memory cases. I'll explain this change and how to avoid the extra memory consumption where appropriate. Second, I'll also show how Analytica's maximum working set size can be configured (on some operating systems), which can be used to keep other applications on your computer highly responsive even when large memory-intensive computations are churning away.

In Analytica 64-bit, you are less likely to encounter the problem of running out of memory, but you still may encounter performance slowdown from huge memory utilization. Some large models, requiring extensive memory resources, run smoothly even when requiring far more memory than there is RAM available, while other result in extensive ''thrashing'', taking very long to evaluate. I've found that differentiating factor has to do with the size of individual arrays within the model. Thrashing tends to occur when a single array consumes more than about 1/3 the available RAM, while if all arrays in the model remain small relative to available RAM, huge memory evaluations often run very smoothly. I'll relate these observations, and what that means with respect to sample size.

This talk is appropriate for advanced Analytica modelers.

=== The [[Dynamic]] Function and Dynamic Loops ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function in Analytica 4.2 is a bit more flexible than previously. It is now possible for [[User-Defined Functions]] to be part of a dynamic loop, and there are cases where it is now possible to operate over the [[Time]] index where previous it was not. I'll discuss how dynamic models are evaluated, and how that differs from non-dynamic models, and cover some tricks for tracing evaluation and debugging complex dynamic models (which can often be challenging to debug). I'll also introduce how [[Dynamic]] can be used on an index other than the built-in [[Time]] index.

=== [[IntraTable]]s ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

An [[IntraTable]] is a variation of an edit [[Table]], where (like [[Table]]s), the cells may contain expressions, but unlike [[Table]]s, the expressions in each cell can refer to values of other cells. This may (quite appropriately) conjure up nightmares of spreadsheet hell. While this isn't something you'd want to use nonchalantly, there are cases when the ability to reference other cells within the same table comes with its benefits. Because the [[IntraTable]] allows any pattern of references (it doesn't have to be strictly left-to-right, for example), as long as a cell-cycle isn't created, it is possible to encode certain recurrences that would otherwise be quite difficult to encode.

== How to Attend ==

To attend, you need to sign up by contacting Lumina at mailto:webinars@lumina.com. Please sign up at least a day prior. Attendance is limited, so please don't sign up unless you sincerely intend to attend.

These Webinars are FREE to users who have an up-to-date Support for Analytica. If you are unsure, check with mailto:sales@lumina. For those without current support, the fee is US$50.

== How to be a Presenter ==

Being a presenter at an Analytica webinar provides an opportunity to make others in the Analytica community aware of your successes or capabilities. Consultants may find this an opportunity for exposure to others with particular modeling needs. Also, if you are an Analytica aficionado, this is a great opportunity to help others.

If you would like to be a presenter, submit your proposed topic to webinars@lumina.com and possible presentation times (include the time zone). We will schedule the GotoMeeting conference (you do not need a gotoMeeting subscription yourself) and we will make you presenter during the session, allowing you to share your screen while you talk. You will most likely make use of Power Point and a running Analytica during your presentation.

==Analytica User Group:([[Analytica_User_Group/Past_Topics#The_Large_Sample_Library|Large Sample Library]])==

== [[Analytica User Group/Past Topics|Archive of Past Topics]] ==

Visit the [[Analytica User Group/Past Topics|Archive of Past Topics]] for abstracts, example models, and recordings of previous Analytica User Group webinars. Previous topics that have been presented include:

* Guidelines for Model Transparency
* Automated Monitoring and Failure Detection
* Principal Component Analysis (PCA)
* Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]])
* Bond Portfolio Analysis
* Net Present Value ([[NPV]])
* The Analytica Wiki, and How to Contribute
* Data Center Capacity Planning
* Modeling the Precision Strike Process
* The Performance Profiler
* Variable Stiffness Cubic Splines
* Introduction to [[DetermTable]]s
* Importance Sampling (Bayesian posteriors)
* Importance Sampling (rare events)
* Statistical Functions
* [[Table Splicing]]
* Analytica Web Player
* [[SubTable|SubTables]]
* Creating Custom Distribution Functions
* Using the Check attribute to validate inputs and results
* The [[Dynamic]] Function
* [[Logistic Regression]]
* Creating Control Panels
* Statistical Functions
* Creating Scatter Plots
* Using [[Regression]]
* New Functions for Reading Directly from an Excel File
* Advanced Tornado Charts -- when inputs are Array-Valued
* Using the Analytica Decision Engine (ADE) from ASP.NET
* OLE Linking
* Tornado Charts
* Correlated and Multivariate Distributions
* Assessment of Probability Distributions
* Graph Style Templates
* Sneak preview of Analytica Web Publisher
* Querying an OLAP server
* Querying an ODBC relational database
* [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]]
* Flattening and Unflattening of Arrays
* Introduction to Arrays and Array Abstraction
* [[Local Indexes]]
* The [[Iterate]] Function
* The [[Using_References|Reference and Dereference Operators]]
* Modeling Utility Tariffs in Analytica
* Modeling Energy Efficiency in Large Data Centers
* Calling External Applications
* Introduction to Linear and Quadratic Programming
* Non-Linear Optimization
* Writing [[User-Defined Functions]]
* Modeling Markov Processes in Analytica
* Manipulating Dates in Analytica
* Button Scripting
* Manipulating Indexes and Arrays in Analytica Expressions
* Edit Table Enhancements in Analytica 4.0
* [[Handle]]s and [[Meta-Inference]]

== Potential future topics ==

If you would like to see a webinar on a given topic, please feel free to add it here. If you see a topic listed and would like to be a presenter, let us know.

* Statistical hypothesis testing.
** Standard textbook tests
** Computing p-values using Monte Carlo for complex and non-standard statistical models.

* Time-series analysis

* Using the interpolation functions, [[LinearInterp]], [[CubicInterp]], [[StepInterp]].

* Numeric precision - numeric round-off, underflow, etc., why they happen, what to do about it.

* Net Present Value and Internal Rate of Return -- introduction to the use of these commonly used metrics for quantifying decision quality.

* Using the ''Performance Profiler'' to understand where your model consumes time and memory resources.

* Producing graphs from ADE: Including how to serve graphs from web pages.

* Intracacies of the Domain attribute

* Mastering Array Abstraction

* Writing User-Defined Distribution functions.

* Getting data into Analytica.

* [[DetermTable]]s

* Large-scale sampling. Techniques when memory limitations constrain sampleSize.

* Creating User-Defined functions. Review of [[Function Parameter Qualifiers]].

* Understanding [[Evaluation Contexts]]

* Bayesian Inference

* Sensitivity Analysis.

* Dynamic Programming

* Approximate and Stochastic Dynamic Programming

== User Survey Results ==

During the first week of September, we sent out a survey to people who had attended webinars so far. Please continue providing us with feedback. Here is some feedback to date:

Level of difficult and speed:

So far, of those who answered this question, 25% say "a bit too easy", 25% say "too hard/fast", and 50% say "just right".

Topics requested for future webinars:

* How to run multiple iterations, e.g. 100 iterations with uncertainty sample of 1000.

* Sampling for rare events.

* Using the lognormal function.

* Financial modeling

* New array functionality (subtables, choices in tables)

* Optimizer !!!

* Dynamic models (twice requested)

* Choice of distributions

* Re-sampling and radomize methods and uncertainty sample size

* Tricks for sensitivity analysis

* Input and output nodes

* Importance analysis

= The Analytica Wiki =

The [[Analytica Wiki ]] contains many resources, including in-depth reference materials, relevant articles, example models, tutorials, etc., to help users master Analytica and find what they need. Even better, Analytica users can contribute! You can upload your own models, articles, expand on or correct materials that are there, etc., for the benefit of the entire Analytica community.

Analytica User Group

2011-03-15T22:19:41Z

Psanford: /* Interactive Optimization Workshop */

[[/Past Topics|Past Topics]] >>

The Analytica User Group is a way for Analytica users to support each other by sharing tips and function libraries. It includes a webinar series with tutorials and demonstrations on key Analytica features.

= Webinar Series =

Analytica User Webinars are a great way to learn about key Analytica features and modeling skills. They are live demos, using Citrix Gotowebinar, and VOIP or your telephone, at your choice. Questions, comments, and tangents are welcome.

There are webinars most weeks, usually on Thursdays at 10am PST (1pm EST). Seats are limited. To sign up for a particular webinar, see "How to Attend" below. Presentations may last anywhere from 20 to 90 minutes (with an estimate provided upfront).

If you missed a User Webinar, don't despair. We usually records them, including audio and screensharing. So, you can replay them any time. See below for the full list.

;Recent talks include:
* [[/Past_Topics#Introduction_to_Structured_Optimization|Introduction to Structured Optimization]] (24 Feb 2011)
* [[/Past_Topics#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]]
** [[/Past_Topics#Session_1:_Uncertainty_and_Probability|Session 1]]: (29 Apr 2010)[[/Past_Topics#Session_1:_Uncertainty_and_Probability|Introduction to Uncertainty, Probability]].
** [[/Past_Topics#Session_2:_Probability_Distributions|Session 2]]: (6 May 2010)[[/Past_Topics#Session_2:_Probability_Distributions|Probabiliity Distributions]]
** [[/Past_Topics#Session_3:_Monte_Carlo|Session 3]]: (13 May 2010)[[/Past_Topics#Session_3:_Monte_Carlo|Monte Carlo Simulation]]
** [[/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Session 4]]: (20 May 2010) [/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Measures of Risk and Utility]]
** [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Session 5]]: (3 June 2010) [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Risk Analysis for Portfolios]]
** [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Session 6]]: (10 June 2010) [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Common Parametric Distributions]]
** [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Session 7]]: (24 June 2010) [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Expert Assessment of Uncertainty]]
** [[/Past_Topics#Session_8:_Hypothesis_Testing|Session 8]]: (15 July 2010) [[/Past_Topics#Session_8:_Hypothesis Testing|Statistical Hypothesis Testing]]
* [[/Past_Topics#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[/Past_Topics#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[/Past Topics#Spearman_Rank_Correlation|Introduction to Spearman's Rank Correlation]] (25 Mar 2010)

== Schedule of Upcoming Webinars ==

=== Interactive Optimization Workshop ===

'''Date and Time:''' Thursday, 24 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Paul Sanford, Lumina Decision Systems

'''Abstract'''
This is an interactive workshop where you will learn the basics of creating Structured Optimization models and challenge yourself to set up and solve some basic examples on your own! No prior training in optimization is required. [Trial Downloads | http://www.lumina.com/products/optimizer-trial/] of Analytica Optimizer 4.3 are now available. Attendees are encouraged to have Analytica Optimizer 4.3 installed and running during the workshop.

=== Optimizing Parameters in a Complex Model to Match Historical Data ===

'''Date and Time:''' Thursday, 31 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Almost all quantitative models have parameters that must be assessed by experts or estimated from historical data. Estimation from historical data can be complicated by the presence of variables that are either unobservable or unavailable in the historical record. Maximum likelihood estimation addresses this by finding the parameter settings that maximize the likelihood of the historical data predicted by the model. In this talk, I will formulate the parameter fitting task as a structured optimization problem (NLP), providing a hands-on demonstration of the new structured optimization features in Analytica 4.3.

=== Optimization with Uncertainty ===

'''Date and Time:''' Thursday, 6 April 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Analytica analyzes uncertainty by conducting a Monte Carlo analysis. When you optimize decision variables in a model containing uncertainty, you have a choice: You can perform one optimization over the Monte Carlo analysis, or you can perform a Monte Carlo sampling of optimizations (i.e., the Monte Carlo is inside the optimization, or the optimization is inside the Monte Carlo). The first case is used when the decision must be taken while the quantities are still uncertain. The second case is used when the values of the uncertain quantities will be resolved before the decisions are taken.

To illustrate, consider the situation faced by a relief organization that provides aid to victims of large natural disasters. In one situation, a decision must be made regarding how many resources to deploy to one particular location that has been hit by a large tsunami. At the time the decision must be made, the number of casualties is highly uncertain. In a different situation, the organization wants to characterize the uncertainty in its need for resources, given that it will optimally deploy resources in response to natural disasters as they occur.

=== Neural Networks ===

'''Date and Time:''' Thursday, 13 April 2011, 10:00am Pacific Daylight Time

=== Linearizing Optimization Models ===

=== [[Model building by mouse]] ===

'''Date and Time:''' (TBD)

'''Presenter''': (TBD)

'''Abstract'''

See Model building by mouse for the concept. The talk will also cover the TemplateInput and TemplateOutput attributes first introduced in 4.2 that make template modules possible, and thus expand the potential power and applicability of this modeling technique.

=== Saving Memory: Controlling when results are (and are not) cached ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

When a variable is computed, Analytica caches the result (i.e., keeps a copy around) so that when the same result is needed later, it does not need to be recomputed. Large array-valued results can end up consuming large amounts of memory, and lead to annoyances such as running out of available memory. There are instances where it isn't really necessary to cache a result -- either it will never be needed a second time, or it isn't a big deal to recompute it, or it is an exact copy of its parent (perhaps because only the result graph settings are different), so that by configuring that variable to not cache its result, or to release its cache as soon as all its children are computed, memory space can be recovered, perhaps allowing larger simulations to be conducted.

Analytica 4.2 introduces a new feature allowing you to configure when results are (or are not) cached. I'll demonstrate how to utilize this feature, and discuss limitations, such as situations where it would be a very bad idea not to cache.

If time allows (it probably will), I'll also throw in two other small memory-related controls new in 4.2. The [[WhatIf]]-style functions ([[WhatIf]], [[WhatIfAll]], [[Dydx]], [[Elasticity]], [[NlpDefine]]), i.e., functions that compute hypotheticals, have been modified in 4.2 to preserve previously computed values of other variables. This is a nice feature, but can impact tight-memory cases. I'll explain this change and how to avoid the extra memory consumption where appropriate. Second, I'll also show how Analytica's maximum working set size can be configured (on some operating systems), which can be used to keep other applications on your computer highly responsive even when large memory-intensive computations are churning away.

In Analytica 64-bit, you are less likely to encounter the problem of running out of memory, but you still may encounter performance slowdown from huge memory utilization. Some large models, requiring extensive memory resources, run smoothly even when requiring far more memory than there is RAM available, while other result in extensive ''thrashing'', taking very long to evaluate. I've found that differentiating factor has to do with the size of individual arrays within the model. Thrashing tends to occur when a single array consumes more than about 1/3 the available RAM, while if all arrays in the model remain small relative to available RAM, huge memory evaluations often run very smoothly. I'll relate these observations, and what that means with respect to sample size.

This talk is appropriate for advanced Analytica modelers.

=== The [[Dynamic]] Function and Dynamic Loops ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function in Analytica 4.2 is a bit more flexible than previously. It is now possible for [[User-Defined Functions]] to be part of a dynamic loop, and there are cases where it is now possible to operate over the [[Time]] index where previous it was not. I'll discuss how dynamic models are evaluated, and how that differs from non-dynamic models, and cover some tricks for tracing evaluation and debugging complex dynamic models (which can often be challenging to debug). I'll also introduce how [[Dynamic]] can be used on an index other than the built-in [[Time]] index.

=== [[IntraTable]]s ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

An [[IntraTable]] is a variation of an edit [[Table]], where (like [[Table]]s), the cells may contain expressions, but unlike [[Table]]s, the expressions in each cell can refer to values of other cells. This may (quite appropriately) conjure up nightmares of spreadsheet hell. While this isn't something you'd want to use nonchalantly, there are cases when the ability to reference other cells within the same table comes with its benefits. Because the [[IntraTable]] allows any pattern of references (it doesn't have to be strictly left-to-right, for example), as long as a cell-cycle isn't created, it is possible to encode certain recurrences that would otherwise be quite difficult to encode.

== How to Attend ==

To attend, you need to sign up by contacting Lumina at mailto:webinars@lumina.com. Please sign up at least a day prior. Attendance is limited, so please don't sign up unless you sincerely intend to attend.

These Webinars are FREE to users who have an up-to-date Support for Analytica. If you are unsure, check with mailto:sales@lumina. For those without current support, the fee is US$50.

== How to be a Presenter ==

Being a presenter at an Analytica webinar provides an opportunity to make others in the Analytica community aware of your successes or capabilities. Consultants may find this an opportunity for exposure to others with particular modeling needs. Also, if you are an Analytica aficionado, this is a great opportunity to help others.

If you would like to be a presenter, submit your proposed topic to webinars@lumina.com and possible presentation times (include the time zone). We will schedule the GotoMeeting conference (you do not need a gotoMeeting subscription yourself) and we will make you presenter during the session, allowing you to share your screen while you talk. You will most likely make use of Power Point and a running Analytica during your presentation.

==Analytica User Group:([[Analytica_User_Group/Past_Topics#The_Large_Sample_Library|Large Sample Library]])==

== [[Analytica User Group/Past Topics|Archive of Past Topics]] ==

Visit the [[Analytica User Group/Past Topics|Archive of Past Topics]] for abstracts, example models, and recordings of previous Analytica User Group webinars. Previous topics that have been presented include:

* Guidelines for Model Transparency
* Automated Monitoring and Failure Detection
* Principal Component Analysis (PCA)
* Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]])
* Bond Portfolio Analysis
* Net Present Value ([[NPV]])
* The Analytica Wiki, and How to Contribute
* Data Center Capacity Planning
* Modeling the Precision Strike Process
* The Performance Profiler
* Variable Stiffness Cubic Splines
* Introduction to [[DetermTable]]s
* Importance Sampling (Bayesian posteriors)
* Importance Sampling (rare events)
* Statistical Functions
* [[Table Splicing]]
* Analytica Web Player
* [[SubTable|SubTables]]
* Creating Custom Distribution Functions
* Using the Check attribute to validate inputs and results
* The [[Dynamic]] Function
* [[Logistic Regression]]
* Creating Control Panels
* Statistical Functions
* Creating Scatter Plots
* Using [[Regression]]
* New Functions for Reading Directly from an Excel File
* Advanced Tornado Charts -- when inputs are Array-Valued
* Using the Analytica Decision Engine (ADE) from ASP.NET
* OLE Linking
* Tornado Charts
* Correlated and Multivariate Distributions
* Assessment of Probability Distributions
* Graph Style Templates
* Sneak preview of Analytica Web Publisher
* Querying an OLAP server
* Querying an ODBC relational database
* [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]]
* Flattening and Unflattening of Arrays
* Introduction to Arrays and Array Abstraction
* [[Local Indexes]]
* The [[Iterate]] Function
* The [[Using_References|Reference and Dereference Operators]]
* Modeling Utility Tariffs in Analytica
* Modeling Energy Efficiency in Large Data Centers
* Calling External Applications
* Introduction to Linear and Quadratic Programming
* Non-Linear Optimization
* Writing [[User-Defined Functions]]
* Modeling Markov Processes in Analytica
* Manipulating Dates in Analytica
* Button Scripting
* Manipulating Indexes and Arrays in Analytica Expressions
* Edit Table Enhancements in Analytica 4.0
* [[Handle]]s and [[Meta-Inference]]

== Potential future topics ==

If you would like to see a webinar on a given topic, please feel free to add it here. If you see a topic listed and would like to be a presenter, let us know.

* Statistical hypothesis testing.
** Standard textbook tests
** Computing p-values using Monte Carlo for complex and non-standard statistical models.

* Time-series analysis

* Using the interpolation functions, [[LinearInterp]], [[CubicInterp]], [[StepInterp]].

* Numeric precision - numeric round-off, underflow, etc., why they happen, what to do about it.

* Net Present Value and Internal Rate of Return -- introduction to the use of these commonly used metrics for quantifying decision quality.

* Using the ''Performance Profiler'' to understand where your model consumes time and memory resources.

* Producing graphs from ADE: Including how to serve graphs from web pages.

* Intracacies of the Domain attribute

* Mastering Array Abstraction

* Writing User-Defined Distribution functions.

* Getting data into Analytica.

* [[DetermTable]]s

* Large-scale sampling. Techniques when memory limitations constrain sampleSize.

* Creating User-Defined functions. Review of [[Function Parameter Qualifiers]].

* Understanding [[Evaluation Contexts]]

* Bayesian Inference

* Sensitivity Analysis.

* Dynamic Programming

* Approximate and Stochastic Dynamic Programming

== User Survey Results ==

During the first week of September, we sent out a survey to people who had attended webinars so far. Please continue providing us with feedback. Here is some feedback to date:

Level of difficult and speed:

So far, of those who answered this question, 25% say "a bit too easy", 25% say "too hard/fast", and 50% say "just right".

Topics requested for future webinars:

* How to run multiple iterations, e.g. 100 iterations with uncertainty sample of 1000.

* Sampling for rare events.

* Using the lognormal function.

* Financial modeling

* New array functionality (subtables, choices in tables)

* Optimizer !!!

* Dynamic models (twice requested)

* Choice of distributions

* Re-sampling and radomize methods and uncertainty sample size

* Tricks for sensitivity analysis

* Input and output nodes

* Importance analysis

= The Analytica Wiki =

The [[Analytica Wiki ]] contains many resources, including in-depth reference materials, relevant articles, example models, tutorials, etc., to help users master Analytica and find what they need. Even better, Analytica users can contribute! You can upload your own models, articles, expand on or correct materials that are there, etc., for the benefit of the entire Analytica community.

Analytica User Group

2011-03-15T22:18:46Z

Psanford: /* Interactive Optimization Workshop */

[[/Past Topics|Past Topics]] >>

The Analytica User Group is a way for Analytica users to support each other by sharing tips and function libraries. It includes a webinar series with tutorials and demonstrations on key Analytica features.

= Webinar Series =

Analytica User Webinars are a great way to learn about key Analytica features and modeling skills. They are live demos, using Citrix Gotowebinar, and VOIP or your telephone, at your choice. Questions, comments, and tangents are welcome.

There are webinars most weeks, usually on Thursdays at 10am PST (1pm EST). Seats are limited. To sign up for a particular webinar, see "How to Attend" below. Presentations may last anywhere from 20 to 90 minutes (with an estimate provided upfront).

If you missed a User Webinar, don't despair. We usually records them, including audio and screensharing. So, you can replay them any time. See below for the full list.

;Recent talks include:
* [[/Past_Topics#Introduction_to_Structured_Optimization|Introduction to Structured Optimization]] (24 Feb 2011)
* [[/Past_Topics#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]]
** [[/Past_Topics#Session_1:_Uncertainty_and_Probability|Session 1]]: (29 Apr 2010)[[/Past_Topics#Session_1:_Uncertainty_and_Probability|Introduction to Uncertainty, Probability]].
** [[/Past_Topics#Session_2:_Probability_Distributions|Session 2]]: (6 May 2010)[[/Past_Topics#Session_2:_Probability_Distributions|Probabiliity Distributions]]
** [[/Past_Topics#Session_3:_Monte_Carlo|Session 3]]: (13 May 2010)[[/Past_Topics#Session_3:_Monte_Carlo|Monte Carlo Simulation]]
** [[/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Session 4]]: (20 May 2010) [/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Measures of Risk and Utility]]
** [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Session 5]]: (3 June 2010) [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Risk Analysis for Portfolios]]
** [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Session 6]]: (10 June 2010) [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Common Parametric Distributions]]
** [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Session 7]]: (24 June 2010) [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Expert Assessment of Uncertainty]]
** [[/Past_Topics#Session_8:_Hypothesis_Testing|Session 8]]: (15 July 2010) [[/Past_Topics#Session_8:_Hypothesis Testing|Statistical Hypothesis Testing]]
* [[/Past_Topics#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[/Past_Topics#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[/Past Topics#Spearman_Rank_Correlation|Introduction to Spearman's Rank Correlation]] (25 Mar 2010)

== Schedule of Upcoming Webinars ==

=== Interactive Optimization Workshop ===

'''Date and Time:''' Thursday, 24 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Paul Sanford, Lumina Decision Systems

'''Abstract'''
This is an interactive workshop where you will learn the basics of creating Structured Optimization models and challenge yourself to set up and solve some basic examples on your own! No prior training in optimization is required. [Trial downloads][http://www.lumina.com/products/optimizer-trial/] of Analytica Optimizer 4.3 are now available. Attendees are encouraged to have Analytica Optimizer 4.3 installed and running during the workshop.

=== Optimizing Parameters in a Complex Model to Match Historical Data ===

'''Date and Time:''' Thursday, 31 March 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Almost all quantitative models have parameters that must be assessed by experts or estimated from historical data. Estimation from historical data can be complicated by the presence of variables that are either unobservable or unavailable in the historical record. Maximum likelihood estimation addresses this by finding the parameter settings that maximize the likelihood of the historical data predicted by the model. In this talk, I will formulate the parameter fitting task as a structured optimization problem (NLP), providing a hands-on demonstration of the new structured optimization features in Analytica 4.3.

=== Optimization with Uncertainty ===

'''Date and Time:''' Thursday, 6 April 2011, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Ph.D., Lumina Decision Systems

'''Abstract'''

Analytica analyzes uncertainty by conducting a Monte Carlo analysis. When you optimize decision variables in a model containing uncertainty, you have a choice: You can perform one optimization over the Monte Carlo analysis, or you can perform a Monte Carlo sampling of optimizations (i.e., the Monte Carlo is inside the optimization, or the optimization is inside the Monte Carlo). The first case is used when the decision must be taken while the quantities are still uncertain. The second case is used when the values of the uncertain quantities will be resolved before the decisions are taken.

To illustrate, consider the situation faced by a relief organization that provides aid to victims of large natural disasters. In one situation, a decision must be made regarding how many resources to deploy to one particular location that has been hit by a large tsunami. At the time the decision must be made, the number of casualties is highly uncertain. In a different situation, the organization wants to characterize the uncertainty in its need for resources, given that it will optimally deploy resources in response to natural disasters as they occur.

=== Neural Networks ===

'''Date and Time:''' Thursday, 13 April 2011, 10:00am Pacific Daylight Time

=== Linearizing Optimization Models ===

=== [[Model building by mouse]] ===

'''Date and Time:''' (TBD)

'''Presenter''': (TBD)

'''Abstract'''

See Model building by mouse for the concept. The talk will also cover the TemplateInput and TemplateOutput attributes first introduced in 4.2 that make template modules possible, and thus expand the potential power and applicability of this modeling technique.

=== Saving Memory: Controlling when results are (and are not) cached ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

When a variable is computed, Analytica caches the result (i.e., keeps a copy around) so that when the same result is needed later, it does not need to be recomputed. Large array-valued results can end up consuming large amounts of memory, and lead to annoyances such as running out of available memory. There are instances where it isn't really necessary to cache a result -- either it will never be needed a second time, or it isn't a big deal to recompute it, or it is an exact copy of its parent (perhaps because only the result graph settings are different), so that by configuring that variable to not cache its result, or to release its cache as soon as all its children are computed, memory space can be recovered, perhaps allowing larger simulations to be conducted.

Analytica 4.2 introduces a new feature allowing you to configure when results are (or are not) cached. I'll demonstrate how to utilize this feature, and discuss limitations, such as situations where it would be a very bad idea not to cache.

If time allows (it probably will), I'll also throw in two other small memory-related controls new in 4.2. The [[WhatIf]]-style functions ([[WhatIf]], [[WhatIfAll]], [[Dydx]], [[Elasticity]], [[NlpDefine]]), i.e., functions that compute hypotheticals, have been modified in 4.2 to preserve previously computed values of other variables. This is a nice feature, but can impact tight-memory cases. I'll explain this change and how to avoid the extra memory consumption where appropriate. Second, I'll also show how Analytica's maximum working set size can be configured (on some operating systems), which can be used to keep other applications on your computer highly responsive even when large memory-intensive computations are churning away.

In Analytica 64-bit, you are less likely to encounter the problem of running out of memory, but you still may encounter performance slowdown from huge memory utilization. Some large models, requiring extensive memory resources, run smoothly even when requiring far more memory than there is RAM available, while other result in extensive ''thrashing'', taking very long to evaluate. I've found that differentiating factor has to do with the size of individual arrays within the model. Thrashing tends to occur when a single array consumes more than about 1/3 the available RAM, while if all arrays in the model remain small relative to available RAM, huge memory evaluations often run very smoothly. I'll relate these observations, and what that means with respect to sample size.

This talk is appropriate for advanced Analytica modelers.

=== The [[Dynamic]] Function and Dynamic Loops ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function in Analytica 4.2 is a bit more flexible than previously. It is now possible for [[User-Defined Functions]] to be part of a dynamic loop, and there are cases where it is now possible to operate over the [[Time]] index where previous it was not. I'll discuss how dynamic models are evaluated, and how that differs from non-dynamic models, and cover some tricks for tracing evaluation and debugging complex dynamic models (which can often be challenging to debug). I'll also introduce how [[Dynamic]] can be used on an index other than the built-in [[Time]] index.

=== [[IntraTable]]s ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

An [[IntraTable]] is a variation of an edit [[Table]], where (like [[Table]]s), the cells may contain expressions, but unlike [[Table]]s, the expressions in each cell can refer to values of other cells. This may (quite appropriately) conjure up nightmares of spreadsheet hell. While this isn't something you'd want to use nonchalantly, there are cases when the ability to reference other cells within the same table comes with its benefits. Because the [[IntraTable]] allows any pattern of references (it doesn't have to be strictly left-to-right, for example), as long as a cell-cycle isn't created, it is possible to encode certain recurrences that would otherwise be quite difficult to encode.

== How to Attend ==

To attend, you need to sign up by contacting Lumina at mailto:webinars@lumina.com. Please sign up at least a day prior. Attendance is limited, so please don't sign up unless you sincerely intend to attend.

These Webinars are FREE to users who have an up-to-date Support for Analytica. If you are unsure, check with mailto:sales@lumina. For those without current support, the fee is US$50.

== How to be a Presenter ==

Being a presenter at an Analytica webinar provides an opportunity to make others in the Analytica community aware of your successes or capabilities. Consultants may find this an opportunity for exposure to others with particular modeling needs. Also, if you are an Analytica aficionado, this is a great opportunity to help others.

If you would like to be a presenter, submit your proposed topic to webinars@lumina.com and possible presentation times (include the time zone). We will schedule the GotoMeeting conference (you do not need a gotoMeeting subscription yourself) and we will make you presenter during the session, allowing you to share your screen while you talk. You will most likely make use of Power Point and a running Analytica during your presentation.

==Analytica User Group:([[Analytica_User_Group/Past_Topics#The_Large_Sample_Library|Large Sample Library]])==

== [[Analytica User Group/Past Topics|Archive of Past Topics]] ==

Visit the [[Analytica User Group/Past Topics|Archive of Past Topics]] for abstracts, example models, and recordings of previous Analytica User Group webinars. Previous topics that have been presented include:

* Guidelines for Model Transparency
* Automated Monitoring and Failure Detection
* Principal Component Analysis (PCA)
* Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]])
* Bond Portfolio Analysis
* Net Present Value ([[NPV]])
* The Analytica Wiki, and How to Contribute
* Data Center Capacity Planning
* Modeling the Precision Strike Process
* The Performance Profiler
* Variable Stiffness Cubic Splines
* Introduction to [[DetermTable]]s
* Importance Sampling (Bayesian posteriors)
* Importance Sampling (rare events)
* Statistical Functions
* [[Table Splicing]]
* Analytica Web Player
* [[SubTable|SubTables]]
* Creating Custom Distribution Functions
* Using the Check attribute to validate inputs and results
* The [[Dynamic]] Function
* [[Logistic Regression]]
* Creating Control Panels
* Statistical Functions
* Creating Scatter Plots
* Using [[Regression]]
* New Functions for Reading Directly from an Excel File
* Advanced Tornado Charts -- when inputs are Array-Valued
* Using the Analytica Decision Engine (ADE) from ASP.NET
* OLE Linking
* Tornado Charts
* Correlated and Multivariate Distributions
* Assessment of Probability Distributions
* Graph Style Templates
* Sneak preview of Analytica Web Publisher
* Querying an OLAP server
* Querying an ODBC relational database
* [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]]
* Flattening and Unflattening of Arrays
* Introduction to Arrays and Array Abstraction
* [[Local Indexes]]
* The [[Iterate]] Function
* The [[Using_References|Reference and Dereference Operators]]
* Modeling Utility Tariffs in Analytica
* Modeling Energy Efficiency in Large Data Centers
* Calling External Applications
* Introduction to Linear and Quadratic Programming
* Non-Linear Optimization
* Writing [[User-Defined Functions]]
* Modeling Markov Processes in Analytica
* Manipulating Dates in Analytica
* Button Scripting
* Manipulating Indexes and Arrays in Analytica Expressions
* Edit Table Enhancements in Analytica 4.0
* [[Handle]]s and [[Meta-Inference]]

== Potential future topics ==

If you would like to see a webinar on a given topic, please feel free to add it here. If you see a topic listed and would like to be a presenter, let us know.

* Statistical hypothesis testing.
** Standard textbook tests
** Computing p-values using Monte Carlo for complex and non-standard statistical models.

* Time-series analysis

* Using the interpolation functions, [[LinearInterp]], [[CubicInterp]], [[StepInterp]].

* Numeric precision - numeric round-off, underflow, etc., why they happen, what to do about it.

* Net Present Value and Internal Rate of Return -- introduction to the use of these commonly used metrics for quantifying decision quality.

* Using the ''Performance Profiler'' to understand where your model consumes time and memory resources.

* Producing graphs from ADE: Including how to serve graphs from web pages.

* Intracacies of the Domain attribute

* Mastering Array Abstraction

* Writing User-Defined Distribution functions.

* Getting data into Analytica.

* [[DetermTable]]s

* Large-scale sampling. Techniques when memory limitations constrain sampleSize.

* Creating User-Defined functions. Review of [[Function Parameter Qualifiers]].

* Understanding [[Evaluation Contexts]]

* Bayesian Inference

* Sensitivity Analysis.

* Dynamic Programming

* Approximate and Stochastic Dynamic Programming

== User Survey Results ==

During the first week of September, we sent out a survey to people who had attended webinars so far. Please continue providing us with feedback. Here is some feedback to date:

Level of difficult and speed:

So far, of those who answered this question, 25% say "a bit too easy", 25% say "too hard/fast", and 50% say "just right".

Topics requested for future webinars:

* How to run multiple iterations, e.g. 100 iterations with uncertainty sample of 1000.

* Sampling for rare events.

* Using the lognormal function.

* Financial modeling

* New array functionality (subtables, choices in tables)

* Optimizer !!!

* Dynamic models (twice requested)

* Choice of distributions

* Re-sampling and radomize methods and uncertainty sample size

* Tricks for sensitivity analysis

* Input and output nodes

* Importance analysis

= The Analytica Wiki =

The [[Analytica Wiki ]] contains many resources, including in-depth reference materials, relevant articles, example models, tutorials, etc., to help users master Analytica and find what they need. Even better, Analytica users can contribute! You can upload your own models, articles, expand on or correct materials that are there, etc., for the benefit of the entire Analytica community.

Using SetContext to efficiently solve NLPs

2011-02-26T11:10:23Z

Psanford: /* Using SetContext */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

== Resolving inefficiency with SetContext ==

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, each context array is first computed and then sliced along all the extrinsic indexes, extracting the particular slice required by the NLP.

But not all arrays containing extrinsic indexes are appropriate candidates for the SetContext parameter. Examining the influence diagram, it is easy to distinguish between nodes that are computed multiple times during the optimization (those downstream from the Decision node) and nodes that are computed only once (all others). Since extrinsic indexes come from outside the optimization, they are invariably introduced by arrays that are only computed once. These arrays are appropriate candidates for the SetContext parameter. Arrays downstream of the optimization are not appropriate candidates because limiting their computation can change the dynamics of the iteration and alter the final result.

In this example we observed that <code>Excess_deaths</code> and <code>Total_cost</code> contain the extrinsic index <code>Location</code>. But this is due to the influence of <code>Population_exposed</code> which is not downstream of the decision node. Setting context on an array limits its computation to the slice required by the optimization. This will also reduce the dimensionality of dependent nodes. Setting context on <code>Population_exposed</code> will eliminate the <code>Location</code> index from <code>Excess_deaths</code> and <code>Total_cost</code> without altering the iteration dynamics.

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

== Applying SetContext to uncertain variables ==

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T11:06:37Z

Psanford: /* Resolving inefficiency with SetContext */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

== Resolving inefficiency with SetContext ==

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, each context array is first computed and then sliced along all the extrinsic indexes, extracting the particular slice required by the NLP.

But not all arrays containing extrinsic indexes are appropriate candidates for the SetContext parameter. Examining the influence diagram, it is easy to distinguish between nodes that are computed multiple times during the optimization (those downstream from the Decision node) and nodes that are computed only once (all others). Since extrinsic indexes come from outside the optimization, they are invariably introduced by arrays that are only computed once. These arrays are appropriate candidates for the SetContext parameter. Arrays downstream of the optimization are not appropriate candidates because limiting their computation can change the dynamics of the iteration and alter the final result.

In this example we observed that <code>Excess_deaths</code> and <code>Total_cost</code> contain the extrinsic index <code>Location</code>. But this is due to the influence of <code>Population_exposed</code> which is not downstream of the decision node. Setting context on an array limits its computation to the slice required by the optimization. This will also reduce the dimensionality of dependent nodes. Setting context on <code>Population_exposed</code> will eliminate the <code>Location</code> index from <code>Excess_deaths</code> and <code>Total_cost</code> without altering the iteration dynamics.

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, each context array is first computed and then sliced along all the extrinsic indexes, extracting the particular slice required by the NLP. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T10:57:32Z

Psanford: /* Inefficiency Example */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

= Resolving inefficiency with SetContext =

Examining the influence diagram, it is easy to distinguish between nodes that are computed multiple times during the optimization (those downstream from the Decision node) and nodes that are computed only once (all others). Since extrinsic indexes come from outside the optimization, they are invariably introduced by arrays that are only computed once. These arrays are appropriate candidates for the SetContext parameter. Arrays downstream of the optimization are not appropriate candidates because limiting their computation can change the dynamics of the iteration and alter the final result.

In this example we observed that <code>Excess_deaths</code> and <code>Total_cost</code> contain the extrinsic index <code>Location</code>. But this is due to the influence of <code>Population_exposed</code> which is not downstream of the decision node. Setting context on an array limits its computation to the slice required by the optimization. This will also reduce the dimensionality of dependent nodes. Setting context on <code>Population_exposed</code> will eliminate the <code>Location</code> index from <code>Excess_deaths</code> and <code>Total_cost</code> without altering the iteration dynamics.

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, each context array is first computed and then sliced along all the extrinsic indexes, extracting the particular slice required by the NLP. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T10:43:41Z

Psanford: /* Inefficiency Example */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

Examining the influence diagram, it is easy to distinguish between nodes that are computed multiple times during the optimization (those downstream from the Decision node) and nodes that are computed only once (all others). Since extrinsic indexes come from outside the optimization, they are invariably introduced by arrays that are only computed once. Setting context on these variables will limit their computation to relevant slices without altering the dynamics of the optimization. Furthermore, their limited computation will reduce the dimensionality of nodes that are computed repeatedly. Appropriate nodes for setting context are those that are computed only once. In this example we observed that <code>Excess_deaths</code> and <code>Total_cost</code> contain the extrinsic index <code>Location</code>. But this is due to the influence of <code>Population_exposed</code> which is not downstream of the decision node. Therefore, setting context on <code>Population_exposed</code> will eliminate the unnecessary index from <code>Excess_deaths</code> and <code>Total_cost</code> without altering the iteration dynamics.

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, each context array is first computed and then sliced along all the extrinsic indexes, extracting the particular slice required by the NLP. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T10:40:13Z

Psanford: /* Inefficiency Example */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

Examining the influence diagram, it is easy to distinguish between nodes that are computed multiple times during the optimization (those downstream from the Decision node) and nodes that are computed only once (all others). Since extrinsic indexes come from outside the optimization, they are invariably introduced by arrays that are only computed once. Setting context on these variables will limit their computation to relevant slices without altering the dynamics of the optimization. Furthermore, their limited computation will reduce the dimensionality of nodes that are computed repeatedly. Appropriate nodes for setting context are those that are computed only once. In this example we observed that <code>Excess_deaths</code> and <code>Total_cost</code> contain the extrinsic index <code>Location</code>. But this is due to the influence of <code>Population_exposed</code> which is not downstream of the decision node. Therefore, setting context on <code>Population_exposed</code> will eliminate the unnecessary index from the optimization without altering the iteration dynamics.

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, each context array is first computed and then sliced along all the extrinsic indexes, extracting the particular slice required by the NLP. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T10:37:26Z

Psanford: /* Inefficiency Example */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

Examining the influence diagram, it is easy to distinguish between nodes that are computed multiple times during the optimization (those downstream from the Decision node) and nodes that are computed only once (all others). Since extrinsic indexes come from outside the optimization, they are invariably introduced by arrays that are only computed once. Setting context on these nodes will limit their computation to relevant slices without altering the dynamics of the optimization. Furthermore, their limited computation will reduce the dimensionality of nodes that are computed repeatedly. Appropriate nodes for setting context are those that are computed only once. In this example we observed that <code>Excess_deaths</code> and <code>Total_cost</code> contain the extrinsic index <code>Location</code>. But this is due to the influence of <code>Population_exposed</code> which is not downstream of the decision node. Therefore, setting context on <code>Population_exposed</code> will eliminate the unnecessary index from the optimization without altering the iteration dynamics.

During the search for the solution, <code>Population_exposed</code> is only computed a single time. Since it is not downstream from the decision variable, its initial value is never invalidated, so for <code>Population_exposed</code>, there is actually no wasted computation. Our concern with inefficiency in this example lies solely with <code>Excess_deaths</code> and <code>Total_cost</code>.

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, each context array is first computed and then sliced along all the extrinsic indexes, extracting the particular slice required by the NLP. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T10:14:08Z

Psanford: /* Using SetContext */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

During the search for the solution, <code>Population_exposed</code> is only computed a single time. Since it is not downstream from the decision variable, its initial value is never invalidated, so for <code>Population_exposed</code>, there is actually no wasted computation. Our concern with inefficiency in this example lies solely with <code>Excess_deaths</code> and <code>Total_cost</code>.

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, each context array is first computed and then sliced along all the extrinsic indexes, extracting the particular slice required by the NLP. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T10:09:11Z

Psanford: /* Using SetContext */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

During the search for the solution, <code>Population_exposed</code> is only computed a single time. Since it is not downstream from the decision variable, its initial value is never invalidated, so for <code>Population_exposed</code>, there is actually no wasted computation. Our concern with inefficiency in this example lies solely with <code>Excess_deaths</code> and <code>Total_cost</code>.

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, the value of each context value is first computed, and then the value is sliced along all the extrinsic indexes, extracting the value at the same coordinate along these extrinsic indexes as the NLP itself. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T09:20:08Z

Psanford: /* Inefficiency Example */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

During the search for the solution, <code>Population_exposed</code> is only computed a single time. Since it is not downstream from the decision variable, its initial value is never invalidated, so for <code>Population_exposed</code>, there is actually no wasted computation. Our concern with inefficiency in this example lies solely with <code>Excess_deaths</code> and <code>Total_cost</code>.

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, the value of each context value is first computed, and then the value is sliced along all the extrinsic indexes, extracting the value at the same coordinate along these extrinsic indexes as the NLP inself. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T09:17:10Z

Psanford: /* Inefficiency Example */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region. When solving this, each of the optimizations is solved separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

During the search for the solution, <code>Population_exposed</code> is only computed a single time. Since it is not downstream from the decision variable, its initial value is never invalidated, so for <code>Population_exposed</code>, there is actually no wasted computation. Our concern with inefficiency in this example lies solely with <code>Excess_deaths</code> and <code>Total_cost</code>.

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, the value of each context value is first computed, and then the value is sliced along all the extrinsic indexes, extracting the value at the same coordinate along these extrinsic indexes as the NLP inself. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Using SetContext to efficiently solve NLPs

2011-02-26T09:15:17Z

Psanford: /* Inefficiency Example */

One of the many complications that arise when solving non-linear optimization problems (NLPs) is that the presence of extrinsic indexes may introduce computational inefficiencies when array results within your model are indexed by the extrinsic dimensions, even though any single optimization needs only a single slice of the result. What happens is that while one NLP is being solved, array results are being computed and propagated within the internals of your model, where only a single slice of the result is actually used by the NLP being solved. Even though it may have required substantial computation to obtain, the remainder of the array ends up being thrown away without being used.

The «SetContext» parameter of [[DefineOptimization]] provides a way to avoid this inefficiency.

The inefficiency only arises when you are solving NLPs, so if your problem is linear or quadratic, there is no need to make use of «SetContext». Also, it only arises when extrinsic indexes are present -- in other words, when [[DefineOptimization]] found that extra indexes were present in the objective or constraints and thus array-abstracted over these extra (extrinsic) indexes, returning an array of optimization problems. Thus, if you are dealing with a single NLP, once again there is no need to worry about this. One situation where it does arise naturally is the case where you are performing an optimization that depends on uncertain quantities, such that in the uncertainty mode you end up with a Monte Carlo sample of NLPs. If you don't utilize «SetContext» appropriately, the time-complexity of your model will scale quadratically with sample size rather than linearly.

= Inefficiency Example =

[[image:txc_setcontext.png]]

Consider the following optimization based on the model depicted above:

::<code>[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost )</code>

For a given <code>Emissions_reduction</code>, the computed mid value for <code>Total_cost</code> is indexed by <code>Location</code>. <code>Location</code> becomes an extrinsic index, i.e., [[DefineOptimization]] is array-abstracted across the <code>Location</code> index and returns an array of optimizations, with a separate optimization for each location. With a separate NLP for each location, the solution finds the optimal emissions reduction for each region. When solving this, each of the optimizations is solved separately.

Now, consider what computation actually takes place as each NLP is being solved. First, the NLP corresponding to <code>@Location=1</code> is solved. The way an NLP is solved is that the solver engine proposes a candidate solution, in this case a possible value for <code>Emissions_reduction</code>. This value is "plugged in" as the value for <code>Emissions_reduction</code> and <code>Total_cost</code> is then evaluated. During that evaluation, <code>Population_exposed</code>, <code>Excess_deaths</code> and <code>Total_cost</code> are each computed and each result is indexed by <code>Location</code>. At this point, <code>Total_cost[@Location=1]</code> is extracted and total cost for the remaining regions is ignored (since the NLP for @Location=1 is being solved). The search continues, plugging a new value into <code>Emissions_reduction</code> and re-evaluating <code>Total_cost</code>, each time all the variables between <code>Emissions_reduction</code> and <code>Total_cost</code> are re-evaluated, and each time <code>Excess_deaths</code> and <code>Total_cost</code> needlessly compute their values for all locations, when in reality only the @Location=1 value ends up being used.

During the search for the solution, <code>Population_exposed</code> is only computed a single time. Since it is not downstream from the decision variable, its initial value is never invalidated, so for <code>Population_exposed</code>, there is actually no wasted computation. Our concern with inefficiency in this example lies solely with <code>Excess_deaths</code> and <code>Total_cost</code>.

== More inefficiency with the Monte Carlo ==

Next, suppose we solve the optimization from an uncertainty view. Here, for each random scenario of the uncertain variables, we'll solve a separate optimization. Hence, <code>Run</code> joins <code>Location</code> as an extrinsic index -- we have an array of optimizations, indexed by <code>Location</code> and <code>Run</code>.

When these are solved, each NLP in this 2-D array is solved separately. So, the NLP at <code>[@Location=1,@Run=1]</code> is solved first, with the optimizer engine re-evaluating the model at several candidate <code>Emission_reduction</code> values until it is satisfied that it has converged to the optimal reduction level. Then the NLP at <code>[@Location=1,@Run=2]</code> is solved in the same fashion, etc.

In this model, every variable with the exception of our decision, <code>Emissions_reduction</code>, and the index <code>Location</code>, has uncertainty. So when one particular NLP is actively being solved, a Monte Carlo sample is being computed for each variable in the model, resulting finally in a 2-D Monte Carlo sample for <code>Total_cost</code>. From this 2-D array, the single cell corresponding to the coordinate of the active NLP is extracted and used, the remaining values wasted.

In an ideal world, we would expect the solution time to scale linearly with the number of locations and linearly with the sample size. With the inefficiencies just described, the complexity to solve each individual NLP scales linearly with number of locations and linearly with sample size, but since the number of NLPs also scales linearly with these, our net solution time scales quadratically with each. Hence, if you double the number of locations, your model takes 4 times longer to find the optimal solution. If you increase your sample size from 100 to 1000, it suddenly takes 100 times longer (not 10-times longer) to solve.

= Using SetContext =

The «SetContext» parameter of [[DefineOptimization]] provides the mechanism that allows us to avoid the inefficiency described above.

In your call to [[DefineOptimization]], you may optionally list a set of ''context variables''. When the solution process for each NLP begins, the value of each context value is first computed, and then the value is sliced along all the extrinsic indexes, extracting the value at the same coordinate along these extrinsic indexes as the NLP inself. For example:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Population_exposed )

Suppose we are solving for the optimum Mid-value for the NLP at @Location=1. At the beginning of the solution process, <code>Population_exposed</code> is computed, found to be indexed by <code>Location</code>, and then sliced to the <code>Population_exposed[@Location=1]</code> value. This value is then cached as the value for <code>Population_exposed</code> while the NLP is being solved. Since this value is not indexed by <code>Location</code>, the values computed during the solve for <code>Excess_deaths</code> and <code>Total_cost</code> are never indexed by <code>Location</code>, and the inefficiency is averted.

To avoid inefficiencies from the [[Run]] index, we could list additional context variables:

[[DefineOptimization]]( decision: Emissions_reduction, Maximize: Total_cost,
SetContext:Base_concentration, Threshold, Health_damage_factor,
Population_exposed, Control_cost_factor, value_of_a_life )

Compare the context variables selected here with the diagram. Do any of these depend directly or indirectly on the decision variable? Can you explain why this set of context variables was selected?

= Selecting Context Variables =

Which variable or variables should you select to use as a context variable? Here are some guidelines.

=== No downstream operations on extrinsic indexes ===

When you select <code>X</code> to be a context variable, it is essential that no variable depending on <code>X</code> operates over an extrisic index. The presence of such a variable would alter the logic of your model and produce incorrect results! For example, if <code>Y</code> is defined as <code>[[Sum]](X,Location)</code>, where <code>Location</code> is extrinsic, then you should not select ''X'' as a context variable.

Suppose there are two extrinsic indexes, ''I'' and ''J'', and you know there are no downstream variables operating on ''I'', but you can't say the same about ''J''. You'd like to select ''X'' as a context variable for only the ''I'' index. What can you do? In this case, you can accomplish this with a slight modification to your model, introducing one new variable and modifying the definition of ''X'' slightly:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

Then you list <code>I_ctx</code> in the SetContext parameter.

=== Not downstream from a decision variable ===

Ideally, a context variable should not be downstream of a decision variable (i.e., it should not be influenced, directly or indirectly, by a decision variable's value).

Variables downstream from decision variables are invalidated every time a new search point is plugged it, and thus must be recomputed every time. By selecting variables that are not downstream (as was done in the above example), the sliced value gets cached once for each NLP solved.

Context variables that are downstream from decisions are handled somewhat differently by SetContext. Instead of replacing the value with the appropriate slice, the pared definition of the variable is temporarily surrounded by a [[Slice]] operation. So, for example, if <code>Control_cost</code> is specified as a decision variable, its parsed definition would become (for the duration of the NLP solve):
:<code>[[Slice]]([[Slice]](-Cost_factor*([[Logten]](1-Reduction)), Location,1),Run,1)</code>
The original definition is underlined to illustrate the modification. This is not as efficient as replacing the value by its slice once. Here, the original underlined definition must be evaluated at each search point, resulting in an array, which then slices out the cell of interest, which remains wasteful.

If you are willing to make minor modifications to your model, it should never be necessary to select a context variable downstream from a decision. If you are tempted to do so, you can always use this trick:

:Variable I_ctx := <code>I</code>
:Variable X := <code>(«origDefinition»)[I=I_ctx]</code>

with <code>[[DefineOptimization]](..., SetContext:I_ctx....)</code>.

=== Do not use the Extrinsic Index itself ===

It may seem like the extrinsic index variable itself is the obvious context variable (e.g., <code>Location</code> in the above example). However, that is a dangerous choice. Indexes are often used as [[Table|table indexes]] -- tables expect their table indexes to be real indexes. Changing a table index to a single value will break any tables that use it. Furthermore, there are likely to be other points in your model that operate over the index in various ways. The [[Run]] index is especially troublesome, since all distributions and statistical functions implicitly operate over it, so you should never use [[Run]].

=== Only the maximally downstream context ===

You may have an entire submodel with hundreds of variables, none of which is downstream of any variable being optimized. The result of this submodel is used by a variable that is downstream from decision variables. In this case, only the final result variable from the submodule needs to be listed in «SetContext». While other variables within the submodule could also be listed, there is no reason to do so, since the context is captured in the submodules final result variable, fully accomplishing its goal of preventing the indexes from unnecessarily entering the computations within the optimization search.

=== Do use Parametric Choice variables ===

It is very common for Analytica models to have [[Choice]] inputs, where you can select a single value or select "All" to compare results over multiple scenarios. In a purely parametric input like this, the model never operates over the index -- it essentially computes each "scenario" separately.

When such an input exists, and the user selects "All", the input becomes an index, and appears as an extrinsic index for your optimization -- i.e., you end up with a separate optimization for each scenario. The efficiency issues discussed on this page then come into play.

A parametric choice input is a perfect candidate for a context variable. By definition, your model already works as expected when a single value is selected, so there is no worry about downstream operations over the extrinsic index, and it is also known not to depend on a decision variable. It avoids the gotchas that you may need to worry about if you try to be more clever.

There is no harm in listing Parametric choice variables even when they are set to a single value. SetContext will have no impact in those cases, but will kick in and preserve efficiency when they get changed to "All".

= Special Uses =

Although the primary purpose of SetContext is provide a mechanism for avoid the computational inefficiency of array-abstracted NLPs, there are some additional uses.

=== Tracking Active NLP when debugging ===

When debugging, you may need to focus on what is happening during a particular NLP. For example, perhaps you are displaying [[MsgBox|message boxes]] or a [[ShowProgressBar|progress bar dialog]] and you want to only have it pop up when a particular NLP is active, or have it display information about which NLP is currently being solved. Using the «SetContext» parameter, you can easily track the coordinate of the currently active NLP.

To do this, just introduce a new context variable for each extrinsic index, e.g.:
:Variable Location_ctx := <code>Location</code>
:Variable Run_ctx := <code>Run</code>

Then you can debugging include code in nodes within your nodes within your model such as:
:<code>[[ShowProgressBar]]("Solving", "Location="&Location_ctx&", Run="&Run_ctx, (Run_ctx-1)/sample_size)</code>
or
:<code>[[If]] Location='Los Angeles' [[Then]] [[MsgBox]](Excess_deaths)</code>

= See Also =

* [[DefineOptimization]]
* [[Choice]]

Tutorial videos

2011-02-25T03:44:20Z

Psanford: /* Introduction to Structured Optimization */

New: I've attempted to impose some categorization on the past user group webinar topics.

The most recent talks were:
* [[#Introduction_to_Structured_Optimization|Introduction to Structured Optimization]] (24 Feb 2011)
* [[#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]] (Session 1: 29 Apr 2010, Session 2:6 May 2010, Session 3:13 May 2010, Session 4:20 May 2010, Session 5:3 June 2010), Session 6:10 June 2010, Session 7:24 June 2010, Session 8:15 July 2010.
* [[#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[#Spearman_Rank_Correlation|Spearman Rank Correlation]] (25 Mar 2010)

= Table and Array Topics =

=== The Basics of Analytica Arrays and Indexes ===

''This webinar is continued across two sessions.''

Date and Time (Part 1): January 10, 2008, 10:00 - 11:00 Pacific Standard Time

Date and Time (Part 1, repeat): January 11, 2008, 10:00 - 11:00 Pacific Standard Time

Date and Time (Part 2): January 17, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

This introductory talk introduces the basic concepts of Analytica indexes and multi-dimensional arrays, as well as the basics of Intelligent Array Abstraction. There are several important differences between Analytica arrays compared to multi-dimensional arrays found in other modeling, database, and programming environments. For example, each dimension of an array is associated with an index object, and there is no inherent ordering to the dimensions of a multi-D array. Intelligent array abstraction is perhaps the most powerful feature in Analytica. The session will include a brief description of what array abstraction does, and how you should take advantage of it.

Part 1 focuses on indexes, 1-D arrays and the uses of the [[Subscript/Slice Operator]].

Part 2 focuses on [[:Category:Array Functions|array functions]], multi-D arrays, and the principles and philosphy of arrays in Analytica.

This talk is intended for beginning Analytica modelers, and for people who have been using Analytica without making substantial use of its array features.

A recording of the two sessions can be viewed at (requires Windows Media Player):
* [http://AnalyticaOnline.com/WebinarArchive/2008-01-11-Intro-to-arrays.wmv Intro-to-arrays (Part 1).wmv]
* [http://AnalyticaOnline.com/WebinarArchive/2008-01-17-Intro-to-arrays2.wmv Intro-to-arrays (Part 2).wmv]

An Analytica model containing the examples created during the webinar can be downloaded from [[media:Intro to intelligent arrays.ana|Intro to intelligent arrays.ana]]. During part 1, the [[media:Plane catching decision with EVIU.ana|Plane catching decision with EVIU.ana]] was also used briefly during the webinar.

=== Manipulating Indexes and Arrays in Analytica Expressions ===

Date and Time: Thursday, Aug 9, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this webinar, I will review many of the common operations applied to indexes and arrays from within Analytica expressions, with a particular emphasis on enhancements in this area that are new to Analytica 4.0. I'll review the often used and very powerful [[Subscript]] and [[Slice]] operations, along with the [[Associative_vs._Positional_Indexing|duality of associational and positional indexing]]. I'll introduce newly introduced extensions for positional indexes, such as the @I, A[@I=n], and @[I=n] operations, and extensions that expose positional duals to various previously-existing associational array functions. I will describe the distinction between index and value contexts in Analytica expressions, along with the distinction between a variable's index value, mid value and sample value, how these may differ ([[Self-Indexed Arrays]]), and how we may access each context-value explicitly. I will also introduce slice assignment -- the ability to assign values to individual slices of an array within an algorithm.

The content of this webinar is most appropriate for users with moderate to advanced Analyica model-building experience.

Here is the Analytica model that was created during this talk: [[media:Indexes and Arrays UG2.ANA | "Indexes and Arrays UG2.ANA"]]. (This wouldn't be very interesting for someone who didn't attend, but it contains the examples we tried).

=== Local Variables ===

'''Date and Time:''' Thursday, 23 July 2009, 10:00-11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

I'll explain distinctions between different types of local variables that can be used within expressions. These distinctions are of primary interest for people implementing [[Meta-Inference]] algorithms, since they have a lot to do with how [[Handle]]s are treated. Analytica 4.2 introduces some new distinctions to the types of local variables, designed to make the behavior of local variables cleaner and more understandable. One type of local variable is the [[LocalAlias]], in which the local variable identifier serves as an alias to another existing object. In contrast, there is the [[MetaVar]], which may hold a [[Handle]] to another object, but does not act as an alias. The only local variable option that existed previously, declared using [[Var..Do]], is a hybrid of these two, which leads to confusion when manipulating [[handle]]s. Since [[LocalAlias..Do]] and [[MetaVar..Do]] have very clean semantics, the use of these when writing [[Meta-Inference]] algorithm should help to reduce that confusion considerably. Inside a [[User-Defined Function]], parameters are also instances of local variables, and depending on how they are declared, may behave as a [[MetaVar]] or [[LocalAlias]], so I'll discuss how these fit into the picture, as well as local indexes and local indexes.

This is appropriate for advanced Analytica modelers.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-07-23-Local-Variables.wmv Local-Variables.wmv]. The analytica file from the webinar is at [[media:Local Variables.ana|Local Variables.ana]], where I've also implemented the exercises that I had suggested at the end of the webinar, so you can look in the model for the solutions.

=== Array Concatenation ===

'''Date and Time:''' Thursday, 25 June 2009 10:00am-11:00 Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Array concatenation combines two (or more) arrays by joining them side-by-side, creating an array having all the elements of both arrays. The special case of list-concatenation joins 1-D arrays or lists to create a list of elements that can function as an index. Array concatenation is a basic, and common, form of array manipulation.

The [[Concat]] function has been improved in Analytica 4.2, so that array concatenation is quite a bit easier in many cases, and the [[ConcatRows]] function is now built-in (formerly it was available as a library function).

I'll take you through examples of array concatenation, including cases that have been simplified with the 4.2 enhancements, to help develop your skills at using [[Concat]] and [[ConcatRows]].

This webinar is appropriate for all levels of Analytica modelers.

You can view a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-06-25-Array-Concatenation.wmv Array_Concatenation.wmv]. The model file created during the webinar is: [[media:Array_Concatenation.ana|Array_Concatenation.ana]].

=== Flattening and Unflattening of Arrays ===

Date and Time: January 31, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

On occassion you may need to flatten a multi-dimensional array into a 2-D table. The table could be called a ''relational representation'' of the data. In some circles it is also refered to as a ''fact table''. Or, you may need to convert in the other direction -- expanding, or unflattening a relational/fact table into a multi-dimensional array. In Analytica, the [[MdTable]] and [[MdArrayToTable]] functions are the primary tools for unflattening and flattening. In this session, I'll introduce these functions and how to use them, several examples, and many variations.

The model developed during this talk is at [[media:Flattening_and_Unflatting_Arrays.ana | Flattening_and_Unflatting_Arrays.ana]]. A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-01-31-Array-Flattening.wmv Array-Flattening.wmv]

=== The [[Aggregate]] Function ===

'''Date and Time:''' Thursday, 2 July 2009, 10:00am - 11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Aggregation is the process of transforming an array based on a fine-grain index into a smaller array based on a coarser-grain index. For example, you might map a daily cash stream into monthly revenue (i.e., reindexing from days to months).

This has always been a pretty common operation in Analytica models, with a variety of techniques for accomplishing it, but it has just become more convenient with the [[Aggregate]] function, new to Analytica 4.2.

In the webinar, I'll be demonstrating the use and generality of the [[Aggregate]] function. In the process, it will also be a chance to review a number of other basic intelligent array concepts, including array abstraction, subscripting, re-indexing, etc.

This webinar is appropriate for all levels of Analytica modelers.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-07-02-Aggregate.wmv Aggregate.wmv]. The model file created during this webinar is: [[media:Aggregate_Function.ana|Aggregate Function.ana]].

=== Sorting ===

'''Date and Time:''' Thursday, 6 Aug 2009, 10:00am-11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This webinar will demonstrate the functions in Analytica that are used to sort (i.e. re-order) data -- the functions [[sortIndex]], [[Rank]], and the new to 4.2 [[Sort]]. I'll cover the basics of using these functions, including how they interact with indexes, how to apply them to arrays of data, and their use with array abstraction. I'll then introduce several new 4.2 extensions for handling multi-key sorts, descending options, and case insensitivity.

This webinar is appropriate for all levels of Analytica modelers.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-08-06-Sorting.wmv Sorting.wmv]. The model file created during the webinar is at [[media:Sorting.ana|Sorting.ana]].

=== [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]] ===

Date and Time: January 24, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

Every dimension of an Analytica array is associated with an index object. [[Array Abstraction]] recognizes when two arrays passed as parametes to an operator or function contain the same indexes. These indexes are more commonly defined by a global index object, i.e., an index object that appears on a diagram as a parallelogram node. However, variable and decision nodes can serve as indexes, and can even have a multi-dimensional value in addition to being an index itself. This is refered to as a [[Self-Indexed Arrays|self index]]. If a variable identifier is used in an expression, the context in which it appears always makes it clear whether the identifier is being used as an index, or as a variable with a value. Self-indexes can arise in several ways, which I will cover. In rare cases, when writing an expression, you may need to be aware of whether you intend to use the index value or the context value of a self-indexed variable. I'll discuss these cases, for example in [[For..Do]] loops, and the use of the [[IndexValue]] function.

In some cases, lists may be used in expressions, and when combined with other results, lists can end up serving as an [[Implicit Dimensions|implicit dimension]] of an array. An implicit dimension is a bit different from a full-fledged index since it has not name, and hence no way to refer to it in an expression where an index parameter is expected. Yet most built-in Analytica functions can still be employed to operate over an implicit index. When an implicit index reaches the top level of an expression, it is promoted to be a self-index. I will explain and demonstrate these concepts.

The model developed during this talk is at [[media:Self-Indexes_Lists_and_Implicit_dimensions.ana|Self-Indexes_Lists_and_Implicit_dimensions.ana]]. A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-01-24-Self-Indexes-Implicit-Dims.wmv Self-Indexes-Implicit-Dims.wmv]

=== Introduction to [[DetermTable]]s ===

'''Date and Time:''' 18 September 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A [[DetermTable]] provides an input view like that of an [[Table|edit table]], allowing you to specify values or expressions in each cell for all index combinations; however, unlike a [[Table|table]], the evaluation of a [[DetermTable|determtable]] conditionally returns only selected values from the table. It is called a determtable because it acts as a deterministic function of one or more discrete-valued variables.
You can conceptualize a determtable as a multi-dimensional generalization of a select-case statement found in many programming languages, or as a value that varies with the path down a decision tree.

[[DetermTable]]s can be used to encode a table of utilities (or costs) for each outcome in a probabilistic model. In this usage, they combine very naturally with [[ProbTable]]s (probability tables) for discrete probabilistic models. They are also extremely useful in combination with [[Choice]] pulldowns, allowing you to keep lots of data in your model, but using only a selected part of that for your analysis. This leads to [[Selective Parametric Analysis]], which is often an effective way of coping with memory capacity limitation in high dimensional models.

In this talk, I'll introduce the [[DetermTable]], show how you create one and describe the requirements for the table indexes. The actual "selection" of slices occurs in the table indexes. Not all indexes have to be selectors, but I'll explain the difference and how the domain attribute is used to establish the table index, while the value is used to select the slice. When you define the domain of a variable that will serve as a [[DetermTable]] index, you have the option of defining the domain as an ''index domain''. This can be extremely useful in combination with a [[DetermTable]], so I will cover that feature as well. It is helpful to understand how the functionality a [[DetermTable]] can be replicated using two nodes -- the first containing an [[Table|Edit Table]] and the second using [[Subscript]]. Despite this equivalence, [[DetermTable]] can be especially convenient, both because it simplifies things by requiring one less node, but also because an [[Table|Edit Table]] can be easily converted into a [[DetermTable]].

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-09-18-DetermTables.wmv DetermTables.wmv]. The examples created while demonstrating the mechanics of [[DetermTable]]s is saved here: [[media:DetermTable intro.ana|DetermTable intro.ana]]. Other example models used were the ''2-branch party problem.ana'' and the ''Compression post load calculator.ana'', both distributed in the Example models folder with Analytica, and the [[media:Loan_policy_selection.ANA|Loan policy selection.ana]] model.

=== [[Table Splicing]] ===

'''Date and Time''': Thursday, August 14, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Edit tables, probability tables and determ tables automatically adjust when their index's values are altered. When new elements are inserted into an index, rows (or columns or slices) are automatically inserted, and when elements are deleted, rows (or columns or slices) are deleted from the tables. This process of adjusting tables is referred to as ''[[Table Splicing|splicing]]''.

Some indexes in Analytica may be computed, so that changes to some input variables could result in dramatic changes to the index value, both in terms of the elements that appear and the order of the elements in the index. This creates a correspondence problem for Analytica -- how do the rows after the change correspond to the rows before the change. Analytica can utilize three different methods for determining the correspondence: associative, positional, or flexible correspondence. I'll discuss what these are and show you how you can control which method is used for each index.

When slices (rows or columns) are inserted in a table, Analytica will usually insert 0 (zero) as the default value for the new cells. It is possible, however, to explicit set a default value, and even to set a different default for each column of the table. Doing so requires some typescripting, but I'll take you through the steps.

Using blank cells as a default value, rather than zero, has some advantages. It becomes quickly apparent which cells need to be filled in after index items are inserted, and Analytica will issue a warning message if blank cells exist that you haven't yet filled in. I'll take you through the steps of enabling blank cells by default.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-08-14-Edit-Table-Splicing.wmv Edit-Table-Splicing.wmv]. (Note: There is a gap in the recording's audio from 18:43-27:35).

=== [[StepInterp|Step Interpolation]] ===

'''Date and Time:''' Thursday, 8 April 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[StepInterp]] function is useful in a number of scenarios, including:
* Discretizing a continuous quantity into a set of finite buckets.
* Looking up a value from a "schedule table" (e.g., tax-rate table, depreciation table)
* Mapping from a date to its fiscal year, when the fiscal year starts on an arbitrary mid-year date.
* Mapping from a cumulated value back to the index element/position.
* Performing a "nearest" or "robust" [[Subscript]] or [[Slice]] operation.
* Interpolating value for a relationship that changes in discrete steps

In this webinar, I'll demonstrate how to use the [[StepInterp]] function on several simple examples.

This webinar is appropriate for beginning Analytica modelers and up.

You can watch a recording of this webinar at: [http://analyticaonline.com/WebinarArchive/2010-04-08-StepInterp.wmv StepInterp.wmv]. You can download the model created during this webinar from [[media:Step Interp Intro.ana|Step Interp Intro.ana]].

=== Analytica Web Player ===

'''Date and Time:''' Thursday, August 7, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

The Analytica Web Player (AWP) is scheduled to launch on July 31, 2008. AWP is a subscription service hosted on Lumina's servers. As a subscriber, you can upload your models to the server and send your colleagues a URL so that they can view your model. To view your models, they need only a Flash-enabled web browser. They can browser your model, change inputs, and evaluate results, all from within their web browser.

In this talk we'll cover the available subscription plans, pricing, limitations, and how you sign up. We'll also demonstrate the process of uploading models and sharing these with colleagues.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-08-07-AWP.wmv AWP.wmv].

=== [[SubTable|SubTables]] ===

'''Date and Time:''' Thursday, 31 July 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[SubTable]] function allows a subset of another edit table to be edited by the user as a different view. To the user, it appears as if he is editing any other edit table; however, the changes are stored in the original edit table. The rows and columns can be transformed to other dimensions in the Subtable, with different index element orders, based on [[Subset]] indexes, and with different number formats.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-31-SubTables.wmv SubTables.wmv]. The model file from this webinar is at [[media:SubTable_webinar.ana]].

=== Edit Table Enhancements in Analytica 4.0 ===

Date and Time: Thursday, Aug 2, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this webinar, I will demonstrate several new edit table functionalities in Analytica 4.0, including:
* Insert [[Choice]] drop-down controls in table cells.
* [[Table Splicing|Splicing]] tables based on computed indexes.
* Customizing the [[TableCellDefault|default cell value(s)]].
* Blank cells to catch entries that need to be filled in.
* [[SubTable]]s
* Using different number formats for each column.

This talk is oriented for model builders with Analytica model-building experience.

The Analytica session that existed by the end of the talk is stored in the following model file: [[Media:Edit_Table_Features.ANA | "Edit Table Features.ana"]].

= Modeling Time =

=== Manipulating Dates in Analytica ===

Date and Time: Thursday, Sept. 13, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this talk, I'll cover numerous aspects relating to the manipulation of dates in Analytica. I'll introduce the encoding of dates as integers and the date origin preference. I'll review how to configure input variables, edit tables, or even individual columns of edit tables to accept (and parse) dates as input. I'll cover date number format capabilities in depth, including how to create your own custom date formats, understanding how date formats interact with your computer's regional settings, and how to restrict a date format to a single column only. We'll also see how axis scaling in graphs is date-aware.

Next, we'll examine various ways to manipulate dates in Analytica expressions. This includes use of the new and powerful functions [[MakeDate]], [[DatePart]], and [[DateAdd]], and some interesting ways in which these can be used, for example, to define date sequences. Finally, we'll practice our array mastery by aggregating results to and from different date granularities, such aggregating from a month sequence to a years, or interpolating from years to months.

The model file resulting by the end of the session is available here: [[media:Manipulating Dates in Analytica.ana|Manipulating Dates in Analytica.ana]].

You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-13-Manipulating-Dates.wmv Manipulating Dates.wmv] (Windows Media Player required) Unfortunately, this one seems to have recorded poorly -- the video size is too small. If you magnify it in your media player, it does become readable. Sorry -- I don't know why it recorded like this.

=== The [[Dynamic]] Function ===

'''Date and Time:''' Thursday, 12 June 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function is used for modeling or simulating changes over time, in which values of variables at time t depend on the values of those variables at earlier time points. Analytica provides a special system index named Time that can be used like any other index, but which also has the additional property that it is used by the [[Dynamic]] function for dynamic simulation.

This webinar is a brief introduction to the use of the Dynamic function and to the creation of dynamic models. I'll cover the basic syntax of the [[Dynamic]] function, as well as various ways in which you can refer to values at earlier time points within an expression. Dynamic models result in influence diagrams that have directed cycles (i.e., where you can start at a node, follow the arrows forward and return to where you started), called dynamic loops. Similar ''cyclic dependencies'' are disallowed in non-dynamic influence diagrams.

During the webinar, we'll loop at several simple examples of Dynamic, oriented especially for those of you with little or no experience with using [[Dynamic]] in models. I'll provide some helpful hints for keeping things straight when building dynamic models. For the more seasoned modelers, I'll also try to fold in a few more detailed tidbits, such as some explanation about how dynamic loops are evaluated, and how variable identifiers are interpreted somewhat differently from within dynamic loops.

The model developed (extension of Fibonacci's rabbit growth model) can be downloaded here: [[media:The Dynamic Function.ana|The Dynamic Function.ana]]. A recording of the webinar can be viewed at
[http://AnalyticaOnline.com/WebinarArchive/2008-06-12-Dynamic-Function.wmv Dynamic-Function.wmv].

=== Modeling Markov Processes in Analytica ===

Date and Time: Thursday, Sept. 20, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Matthew Bingham, Principal Economist, Veritas Economic Consulting

Abstract

The class of mathematical processes characterized by dynamic dependencies between successive random variables is called Markov chains. The rich behavior and wide applicability of Markov chains make them important in a variety of applied mathematical applications including population and demographics, health outcomes, marketing, genetics, and renewable resources. Analytica’s dynamic modeling capabilities, robust array handling, and flexible uncertainty capabilities support sophisticated Markov modeling. In this webinar, a Markov modeling application is demonstrated. The model develops age-structured population simulations using a Leslie matrix structure and dynamic simulation in Analytical.

A recording of this session can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2007-09-20-Markov-Processes.wmv Markov-Processes.wmv] (requires Windows Media Player)

An article about the model presented here: [[media:AnalyticaMarkovtext.pdf|AnalyticaMarkovtext.pdf]]

= Analytica Language Features =

=== [[Local Indexes]] ===

Date and Time: Thursday, Dec. 13, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

A [[Local Indexes|local index]] is an index object created during the evaluation of an expression using either the [[Index..Do]] or [[MetaIndex..Do]] construction. Local indexes may exist only temporarily, being reclaimed when they are no longer used, or they may live on after the evaluation of the expression has completed, as an index of the result. Some operations require the use of local indexes, or otherwise could not be expressed.

In this talk, I'll introduce simple uses of local indexes, covering how they are declared using [[Index..Do]], with several examples. We'll see how to access a local index using the [[Dot operator::A.I|A.I]] operator. I'll discuss the distinctions between local indexes and local variables. I'll show how the name of a local index can be computed dynamically, and I'll briefly cover the [[IndexNames]] and [[IndexesOf]] functions.

The model created during this talk is here: [[media:Webinar_Local_Indexes.ana|Webinar_Local_Indexes.ana]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-12-13-Local-Indexes.wmv Local-Indexes.wmv] (Requires Windows Media Player)

=== [[Handle]]s and [[Meta-Inference]] ===

Date and Time: Thursday, Dec. 6, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

Meta-inference refers to computations that reason about your model itself, or that actually alter your model. For example, if you were to write an expression that counted how many variables are in your model, you would be reasoning about your model. Other examples of meta inference include changing visual appearance of nodes to communicate some property, re-arranging nodes, finding objects with given properties, or even creating a transformed model based on portion of your model's structure.

The ability to implement meta-inferential algorithms in Analytica has been greatly enhanced in Analytica 4.0. The key to implementation of meta-inference is the manipulation of [[Handle]]s to objects (formerly refered to as ''varTerms''). This webinar will provide a very brief introduction to handles and using them from within expressions. I will assume you are pretty familiar with creating models and writing expressions in Analyica, but I will not assume that have previous seen or used Handles. This topic is oriented towards more advanced Analytica users.

The model used/created during this webinar as at: [[media:Handle and MetaInference Webinar.ANA|Handle and MetaInference Webinar.ANA]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-12-06-Handles.wmv Handles.wmv] (Requires Windows Media Player)

=== The [[Iterate]] Function ===

Date and Time: Thursday, Nov. 29, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

With [[Iterate]], you can create a recurrent loop around a large model, which can be useful for iterating until a convergence condition is reached, for example. For complex iterations, where many variables are being updated at each iteration, requires you to structure your model appropriate, bundling and unbundling values within the single iterative loop. With some work, [[Iterate]] can be used to simulate the functionality [[Dynamic]], and thus provides one option when a second [[Time]]-like index is needed (although not nearly as convenient as [[Dynamic]]).

In this session, we'll explore how [[Iterate]] can be used.

Here is the model file developed during the webinar: [[media:Iterate Demonstration.ANA|Iterate Demonstration.ANA]]

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-29-Iterate.wmv Iterate.wmv] (Requires Windows Media Player)

=== The [[Using_References|Reference and Dereference Operators]] ===

Date and Time: Thursday, Nov. 15, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract
The reference operators make it possible to represent complex data structures like trees or non-rectangular arrays, bundle heterogenous data into records, maintain arrays of local indexes, and seize control of array abstraction in a variety of scenarios. Using a reference, an array can be made to look like an atomic element to array abstraction, so that arrays of differing dimensionality can be bundled into a single array without an explosion of dimensions. The flexibilities afforded by references are generally for the advanced modeler or programmer, but once mastered, they come in useful fairly often.

Here is the model used during the webinar: [[media:Webinar Reference and Dereference Operators.ANA|Reference and Dereference Operators.ana]].
Near the end of the webinar, I encountered a glitch that I was not able to resolve until after the webinar was over. This has been fixed in the attached model. For an explanation of what was occurring, see:
[[Analytica_User_Group/Reference_Webinar_Glitch]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-15-Reference-And-Dereference.wmv Reference-And-Dereference.wmv] (Requires Windows Media Player)

=== Writing [[User-Defined Functions]] ===

Date and Time: Thursday, Sept. 27, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

When you need a specialized function that is not already built into Analytica, never fear -- you can create your own [[User-Defined Functions|User-Defined Function (UDF)]]. Creating UDFs in Analytica is very easy. I'll introduce this convenient capability, and demonstrate how UDFs can be organized into libraries and re-used in other models. I'll also review the libraries of functions that come with Analytica, providing dozens of additional functions.

After this introduction to the basics of UDFs, I'll dive into an in-depth look at [[Function Parameter Qualifiers]]. There is a deep richness to function parameter qualifiers, mastery of which can be used to great benefit. One of the main objectives for a UDF author, and certainly a hallmark of good modeling style, should be to ensure that the function fully array abstracts. Although this usually comes for free with simple algorithms, it is sometimes necessary to worry about this explicitly. I will demonstrate how this objective can often be achieved through appropriate function parameter qualification.

Finally, I will cover how to write a custom distribution function, and how to ensure it works with [[Mid]], [[Sample]] and [[Random]].

This talk is appropriate for Analytica modelers from beginning through expert level. At least some experience building Analytica models and writing Analytica expressions is assumed.

The model created during this webinar, complete with the UDFs written during that webinar, can be downloaded here: [[media:Writing User Defined Functions.ana|Writing User Defined Functions.ana]].

You can watch this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-27-Writing-UDFs.wmv Writing-UDFs.wmv] (Windows Media Player required)

=== Custom Distribution Functions ===

'''Date and Time:''' Thursday, 24 July 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Analytica comes with most of the commonly seen [[:category:Distribution Functions|distributions]] built-in, and many additions [[:category:Distribution Functions|distribution functions]] available in the standard libraries. However, in specific application areas, you may encounter distribution types that aren't already provided, or you may wish to create a variation on an existing distribution based on a different set of parameters. In these cases, you can create your own ''User-Defined Distribution Function'' (UDDF). Once you've created your function, you can utilize it within your model like you would any other distribution function.

User-defined distribution functions are really just instances of [[User-Defined Functions]] (UDFs) that behave in certain special ways. This webinar discusses the various functionalities that a user-defined distribution function should exhibit and various related considerations. Most fundamentally, the defining feature of a UDDF is that it returns a median value when evaluated in Mid mode, but a sample indexed by Run when evaluated from Sample mode. This contrasts with non-distribution functions whose behavior does not depend on the Mid/Sample evaluation mode. Custom distributions are most often implemented in terms of existing distributions (which includes Inverse CDF methods for implementing distributions), so that this property is achieved automatically since the existing distributions already have this property. But in less common cases, UDDFs may treat the two evaluation modes differently.

When you create a UDDF, you may also want to ensure that it works with [[Random]]() to generate a single random variate, and supports the [[Distribution Functions#The Over parameter|Over parameter]] for generating independent distributions. You may also want to create a companion function for computing the density (or probability for discrete distributions) at a point, which may be useful in a number of contexts including, for example, during importance sampling. I'll show you how these features are obtained.

There are several techniques that are often used to implement distribution functions. The two most common, especially in Analytica, are the ''Inverse CDF'' technique and the ''transformation from existing distributions'' method. I'll explain and show examples of both of these. The Inverse CDF is particularly convenient in that it supports all sampling methods (Median Latin Hypercube, Random Latin Hypercube, and Monte Carlo).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-24-Custom-Distribution-Functions.wmv Custom-Distribution-Functions.wmv]. The model file created during the webinar is [[media:Custom Distribution Functions.ana|Custom Distribution Functions.ana]].

=== [[Regular Expressions]] ===

'''Date and Time:''' Thursday, 9 July 2009, 10:00am - 11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Analytica 4.2 exposes a new and powerful ability to utilize Perl-Compatible regular expressions for text expression analysis. This feature has particular applicability for parsing application when importing data. Long known as the feature that makes Perl and Python popular as data file processing languages, that same power is now readily available within Analytica's [[FindInText]], [[SplitText]], and [[TextReplace]] functions.

This talk will only touch on the regular expression language itself (information on which is readily available elsewhere), but instead focuses on the use of these expressions from the Analytica expressions, especially the extracting of text that matches to subpatterns and finding repeated matches.

One relatively complex example that I plan to work through is the parsing of census population data from datafiles downloaded from the U.S. census web site. The task includes parsing highly variable HTML, as well as multiple CSV files with formatting variations that occur from element to element. These variations, which are typical in many sources of data, demonstrate why the flexibility of regular expressions can be extremely helpful when parsing data files.

Regular expressions themselves are extremely powerful, but when overused, can be very cryptic. So even though it is possible to get carried away with this power, it is good to know how to balance the temptation.

This talk is appropriate for moderate to advanced level modelers.

A recording of this webinar can be watched at [http://AnalyticaOnline.com/WebinarArchive/2009-07-09-Regular-Expressions.wmv Regular-Expressions.wmv]. If you are new to regular expressions, I've included a slides on the regular expression patterns that I made use of in [[media:Regular_Expressions.pps|this power point show]] (these were not shown during the webinar). The model file developed during the webinar is [[media:Regular expressions.ana|Regular expressions.ana]].

=== Using the [[Check Attribute]] to validate inputs and results ===

''' Date and Time:''' Thursday, 17 July 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Check Attribute|check attribute]] provides a way to validate inputs and computed results. When users of your model are entering data, this can provide immediate feedback when they enter values that are out of range or inconsistent. When applied to computed results, it can help catch inconsistencies, which can help reduce error rates and accidental introduction of errors later.

In this talk, I'll demonstrate how to define a check validation for a variable, and how to turn on the check attribute visible so that it is visible in the object window. I'll demonstrate how the failed check alert messages can be customized. And perhaps most interestingly, how the check can be used in edit tables for cell-by-cell validation, so that out-of-range inputs are flagged with a red background, and alert balloons pop-up when out-of-range inputs are entered. Cell-by-cell validation when certain restrictions on the check expression are followed, which I'll discuss.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-17-Check-Attribute.wmv Check-Attribute.wmv] (Note: There is audio, but screen is black, for first 50 seconds). The model used during this webinar, with the check attributes inserted, is at [[media:Check attribute -- car costs.ana|Check attribute -- car costs.ana]].

=== The Performance Profiler ===

'''Date and Time:''' October 9, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''Requires Analytica Enterprise''

When you have a model that takes a long time to compute, thrashes in virtual memory, or uses up available memory, the Performance Profiler can tell you where your model is spending its time and how much memory is being consumed by each variable to cache results. It is not uncommon to find that even in a very large model, a small number (e.g., 2 to 5) of variables account for the lion's share of time and memory. With this knowledge, you can focus your attention optimizing the definition of those few variables. On several occassions I've achieved more than 100-fold sped up in computation time on large models using this technique.

The Performance Profiler requires with Analytica Enterprise or Optimizer. I'll demonstrate how to use the profiler with some basic discussions of what is does and does not measure. One neat aspect of the profiler is that you can actually activate it after the fact. In otherwords, even though you haven't adding profiling to your model, if you happen to notice something taking a long time, you can add it in to find out where the time was spent.

Using the Profiler is pretty simple, so I expect this session will be somewhat shorter than usual. The content will be oriented primarily to people who are unfamiliar with the profiler, although I will also try to provide some behind the scenes details and can answer questions about it for

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-10-09-Performance-Profiler.wmv Performance-Profiler.wmv]. The model file containing the first few examples from the webinar can be downloaded from [[media:Simple Performance Profiler Example.ana|Simple Performance Profiler Example.ana]].

= Organizing Models =

=== Modules and Libraries ===

'''Date and Time:''' 10 Dec 2009 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Modules form the basic organizational principle of an Analytica model, allowing models to be structured hierarchically, keeping things simple at every level even in very large complex models. You can use linked modules to store your model across multiple files. This capability enables reuse of libraries and model logic across different models, and allows you to divide your model into separate pieces so that different people can work concurrently on different pieces of the model.

In this talk, I will review many aspects of modules and libraries. We'll see how to use linked modules effectively. I'll cover what the the distinctions are between Modules, Libraries, Models and Forms. I'll demonstrate various considerations when adding modules to existing models -- such as whether you want to import system variables or merge (update) existing objects, and some variations on what is possible there. We'll see how to change modules (or libraries) from being embedded to linked, or vise versa, and how to change the file location for a linked module. When distributing a model consisting of multiple module files, I'll go over directory structure considerations (the relative placement of module files), and also demonstrate how you can store a copy of your model with everything embedded in a single file for easy distribution.

I'll also discuss definition hiding and browse-only locking. By locking individual modules, you can create libraries with hidden and unchangeable logic that can be used in the context of other people's models, keeping your algorithms hidden. Or, you can distribute individual models that are locked as browse only, even in the context of a larger model where the remainder of the model is editable.

I'll talk about using linked modules in the context of a source control system, which is often of interest for projects where multiple people are modifying the same model. I'll also reveal an esoteric feature, the Sys_PreLoadScript attribute, and how this can be used to implement your own licensing and protection of intellectual property.

This webinar is appropriate for all levels of Analytica model builders.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-12-10-Linked-Modules.wmv Linked-Modules.wmv]. The starting model used in the webinar can be downloaded from [[media:Loan_policy_selection_start.ana|Loan_policy_selection_start.ana]], and then you can follow along to introduce and adjust modules as depicted in the recording if you like.

= Uncertainty & Probability Topics =

=== Gentle Introduction to Modeling Uncertainty: Webinar Series ===

'''Date and Time:'''
:'''Session 1:''' Thursday, 29 Apr 2010 10:00am Pacific Daylight Time
:'''Session 2:''' Thursday, 6 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Are you someone who has never built a model containing explicit representions of uncertainty? Did that Statistics 1A class you took a long time ago instill a belief that probability distributions are irrelevant to the type stuff you work on? Are you afraid to start representing uncertainty explicitly because you just don't have the statistics background and don't know much about probability and probability distributions?

If any of these sentiments resonate with you, then this webinar (series) may be for you.

These are interactive webinars. Be prepared to answer some questions, and have Analytica fired up in the background. You are going to use it to compute the answer to a couple exercises! Even if you are watching the recording, be ready to complete the exercises.

This webinar series is most appropriate for:
* Beginning Analytica model builders.
* Users of models that present results with uncertainty.
* Accomplished spreadsheet or Analytica model builders who have not previously incorporated uncertainty.
* People looking to learn the basics of probability for the representation of uncertainty.

==== Session 1: Uncertainty and Probability ====

In the first session discusses different sources and types of uncertainty, probability distributions and how they can be used to represent uncertainty, various different interpretations of probabilities and probability distributions, and reasons why it is valuable to represent uncertainty explicitly in your quantitative models.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2010-04-29-Modeling-Uncertainty1.wmv Modeling-Uncertainty1.wmv]. A copy of the model created by the presenter during the webinar (the scholarship example) can be downloaded from [[media:Modeling uncertainty 1 - princeton scholarship.ana|Modeling uncertainty 1 - princeton scholarship.ana]]. Power point slides can be downloaded from: [[media:Modeling Uncertainty 1.ppt|Modeling Uncertainty 1.ppt]].

==== Session 2: Probability Distributions ====

How do you characterize the amount of uncertainty you have regarding a real-valued quantity? This second session explores this question, and introduces the concepts of average deviation (aka absolute deviation), variance and standard deviation. It then introduces the concept of a ''probability distribution'' and the [[Normal]] and [[LogNormal]] distributions. We examine the [http://en.wikipedia.org/wiki/EVIU expected value of including uncertainty] and do a few modeling exercises that demonstrate how it can be highly misleading, even expensive, to ignore uncertainty.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-06-Prob-Distributions.wmv Prob-Distributions.wmv]. The model build during the webinar can be downloaded from [[media:Probability Distributions Webinar.ana|Probability Distributions Webinar.ana]]. Power point slides are at [[media:Modeling Uncertainty 2.ppt|Modeling Uncertainty 2.ppt]].

==== Session 3: Monte Carlo ====

'''Date and Time:''' Thursday, 13 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

In this third webinar in the "Gentle Introduction to Modeling Uncertainty" series, we will see how a probability distribution can be represented as a set of representative samples, and how this leads to a very general method for propagating uncertainty to computed results. This method is known as Monte Carlo simulation.

Analytica represents uncertainty by storing a representative sample, so we'll be learning about how Analytica actually carries out uncertainty analysis. We explore how all the uncertainty result views in Analytica are created from the sample, and learn various 'tricks' for nice histograms for PDF views in various situations.

We'll learn about the [[Run]] index, and how this places samples across different variables in correspondence. We'll learn about the generality of the Monte Carlo for propagating uncertainty, and also learn what Latin Hypercube sampling is.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-13-Monte-Carlo.wmv Monte-Carlo.wmv]. The power point slides are at: [[media:Monte Carlo Simulation.ppt|Monte Carlo Simulation.ppt]]. Example models created during the webinar include: [[media:Representing Uncertainty - Mining Example.ana|Mining Example.ana]], [[media:Representing Uncertainty - Explicit samples.ana|Explicit samples.ana]]and [[media:Representing Uncertainty 3 - Misc.ana|Representing Uncertainty 3 - Misc.ana]] (product of normals and comparison between Latin Hypercube and Monte Carlo precision).

==== Session 4: Measures of Risk and Utility ====

'''Date and Time:''' Thursday, 20 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This fourth webinar in the "Gentle Introduction to Modeling Uncertainty" series will explore concepts and quantitative measures of Risk and Utility. We'll discuss various conceptions and types of risk, and explore topics relevant to model-building that include utility and loss functions, expected value, expected utility, risk neutrality, risk aversion, fractiles and Value-at-risk (VaR).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-20-Risk-And-Utility.wmv Risk-And-Utility.wmv]. The power point slides can be viewed at [[media:Measures of Risk and Utility.ppt|Measures of Risk and Utility.ppt]]. There is an interesting modeling exercise and exploration of Expected Shortfall near the end of the power point slides that was not covered during the webinar. The worked out model examples from the webinar, along with a solution to the final example not covered, can be downloaded from [[media:Measures of Risk.ana|Measures of Risk.ana]].

==== Session 5: Risk Analysis for Portfolios ====

'''Date and Time:''' Thursday, 3 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Committing to a single project or investing in a single asset entails a certain amount of risk along with the potential payoff. If you are able to proceed with multiple projects or invest in multiple assets, the degree of risk may be reduced substantially with small impact on potential return. In this fifth webinar in the "Gentle Introduction to Modeling Uncertainty" series, we'll look at modeling portfolios, such as portfolios of investments or portfolios of research and development projects, and the impact this has on risk and return. Portfolio analysis is the basis for practices such as diversification and hedging, and is a key of risk management.

As with other topics in this webinar series, the presentation and discussion is designed for people who are new to the use of these concepts in a model building context.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2010-06-03-Portfolio-Risk.wmv Portfolio-Risk.wmv]. The Power Point slides are at [[media:Risk Analysis for Portfolios.ppt|Risk Analysis for Portfolios.ppt]]. These include some exercises at the end (for homework!) not covered during the webinar, including continuous portfolio allocations. The model developed during the webinar, augmented to include answers to additional exercises is at [[media:Risk Analysis for Portfolios.ana|Risk Analysis for Portfolios.ana]].

==== Session 6: Common Parametric Distributions ====

'''Date and Time:''' Thursday, 10 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

During the first five sessions of the ''Gentle Introduction to Modeling Uncertainty'' webinar series, you have been introduced to three distribution functions: [[Bernoulli]], [[Normal]] and [[LogNormal]]. In this webinar, we're going to increase this repertoire and learn about other common parametric distributions. I'll discuss situations where specific distributions are particularly convenient or natural for expressing uncertainty about certain types of quantities, and other reasons for why you might prefer one particular distribution type over another. We'll also examine the distinction between discrete and continuous distributions.

As with other topics in this webinar series, the presentation and discussion is designed for people who are new to the use of these concepts in a model building context.

A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-06-10-Parametric-Distributions.wmv Parametric-Distributions.wmv]. The power point slides are at [[media:Common Parametric Distributions.ppt|Common Parametric Distributions.ppt]], and the Analytica model containing the exercises and solutions to exercises not covered during the live recording is at [[media:Common-Parametric-Distributions.ana|Common-Parametric-Distributions.ana]].

==== Session 7: Expert Assessment of Uncertainty ====

'''Date and Time:''' Thursday, 24 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

For most uncertainty analysis, uncertainties about many key quantities must be assessed by expert judgment. There has been a lot of empirical research on human abilities to express their knowledge and uncertainty in the form of probability distributions. It shows that we are liable to a variety of biases, such as overconfidence and motivational biases. I'll give an introduction to practical methods developed by decision analysts to avoid or minimize these biases. I'll give some examples from recent work in expert elicitation for the Department of Energy on the future performance of renewable energy technologies.
I'll also discuss ways to aggregate judgments from different experts.

The session is appropriate for people who are new to this area. This probably includes just about everybody!

This session will draw from Chapters 6 and 7 of "Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis" by M Granger Morgan & Max Henrion, Cambridge University Press, 1992

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2010-06-24-Expert-Assessment.wmv Expert-Assessment.wmv]. (At this time, there seems to be a problem with the uploaded file. We'll attempt to re-upload it, hopefully on Fri 16 July 2010).

==== Session 8: Hypothesis Testing ====

'''Date and Time:''' Thursday, 15 July 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Hypothesis testing from classical statistics addresses the question of whether the apparent support for a given hypothesis is statistically significant. In the field of classical statistics, this is perhaps the most heavily emphasized application of probability concepts, and the methodology is used (if not required by editors) when publishing results for research studies in nearly every field of empirical study.

To illustrate the basic idea, suppose a journalist selects 10 Americans at random and asks whether they support a moratorium on deep sea drilling. Seven of the 10 respond "yes", so the the next day he publishes his article "The Majority of Americans Support a Moratorium on Deep Sea Drilling". His sample is certainly consistent with this hypothesis, but his conclusion is not credible because with such a small sample, this majority could have easily been a random quirk (sampling error). Hence we would say that the conclusion is not "''statistically significant''". But how big does the sample have to be to achieve statistical significance? Where should we draw the line when determining whether the data's support is statistically significant? These are the types of questions addressed by this area of statistics.

Hypothesis testing is a central topic in every introductory Statistics 1A course, often comprising more than half of the total course syllabus. But most introductory courses emphasize a cookbook approach in favor of a conceptual understanding, apparently in the hope of providing people in non-statistical fields step-by-step recipes to follow when they need to publish results in their own fields. As a result, the methodology is possibly misused more often than it is applied correctly, and published results are commonly misinterpreted.

In this seminar, I intend to emphasize a conceptual understanding of the statistical hypothesis methodology rather than the more traditional textbook methodology. After this webinar, when you read "our hypothesis was confirmed by the data at a p-value=0.02 level", or "the hypothesis was rejected with a p-value of 0.18", you should be able to precisely relay what these statements really do or do not imply. You should understand what a p-value and confidence level really denote -- they do not represent, as many people think, the probability that the hypothesis is true.

We will also, of course, examine how we can carry out computations of significance levels (i.e., p-values) within Analytica. Statistics texts are filled with numerous "standard" hypothesis tests (e.g., t-tests, etc), each based on a specific set of assumptions. In this webinar, we'll dive into this in a more general way, where we get to start with our own set of arbitrary assumptions, leveraging the power of Monte Carlo for computation. This means there are no recipes to remember, you can compute significance levels for any statistical model, even if the same assumptions don't appear in your statistics texts, and most importantly, you'll be left with a more general understanding of the concepts.

As a prerequisite, this webinar will assume little more than the introductory background from the earlier webinars in this "Gentle Introduction to Modeling Uncertainty" series. It is appropriate for people who have never taken a Statistics 1A course, or for the majority of people who have taken that introduction to Statistics but could use a refresher.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2010-07-15-Hypothesis-Testing.wmv Hypothesis-Testing.wmv]. To follow along with the webinar, you'll want to also download the Analytica model file [[media:Hypothesis Test S&P Volatility.ana|Hypothesis Test S&P Volatility.ana]] before staring. You'll use the data in that model for the various exercises during the webinar.

Solutions to exercises are saved in this version of the model (created during the webinar): [[media:Hypothesis Test S&P Volatility solution.ana|Hypothesis Test S&P Volatility solution.ana]]. I also inserted a solution to the Parkinson's data test that wasn't covered in the webinar but is contained in the [[media:Hypothesis Testing.ppt|Power Point slides]].

=== Correlated and Multivariate Distributions ===

'''Date and Time:''' Thursday, March 13, 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This talk will discuss various techniques within Analytica for defining probability distributions with specified marginal distributions, and also being correlated with other uncertain variables. Techniques include the use of conditional and hierarchical distributions, multivariate distributions, and Iman-Conover rank-correlated distributions.

The model created during session talk is [[media:Correlated distributions.ana|Correlated distributions.ana]]. You can watch a recording of the webinar from [http://AnalyticaOnline.com/WebinarArchive/2008-03-13-Correlated-Distributions.wmv Correlated-Distributions.wmv].

=== Assessment of Probability Distributions ===

'''Date and Time:''' March 6, 2008 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

''' Abstract'''

When building a quantitative model, we usually need to come up with estimates for many of the parameters and input variables that we use in the model. Because these are estimates, it is good idea to encode these as probability distributions, so that our degree of ''subjective uncertainty'' is explicit in the model. The process of encoding a distribution to reflect the level of knowledge that you (or the experts you work with) have about the true value of the quantity is referred to as ''probability (or uncertainty) assessment'' or ''probability elicitation''.

This webinar will be a highly interactive one, where all attendees are expected to participate in a series of uncertainty assessments as we explore the effects of cognitive biases (such as over-confidence and anchoring), understand what it means to be ''well-calibrated'', and utilize scoring metrics to measure your own degree of calibration. These exercises can help you improve the quality of your distribution assessments, and serve as tools that can help you to when eliciting estimates of uncertainty from other domain experts.

The Analytica model [[media:Probability assessment.ana|Probability assessment.ana]] contains a game of sorts that takes you through several probability assessments and scores your responses. Participants of the webinar played this game by running this model, if you are going to watch the webinar, you will want to do the same. You may want to wait until the appropriate point in the webinar (after preliminary stuff has been covered) before starting. You can watch the webinar recording here: [http://AnalyticaOnline.com/WebinarArchive/2008-03-06-Probability-Assessment.wmv Probability-Assessment.wmv].
The power point slides from the talk are here: [[media:Assessment_of_distributions.ppt|Assessment_of_distributions.ppt]].

=== Statistical Functions ===

'''Date and Time:''' Thursday, 21 Aug 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''This topic was presented in Aug 2007, but not recorded at that time.''

A statistical function is a function that process a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica. I'll describe several built-in statistical functions such as [[Mean]], [[SDeviation]], [[GetFract]], [[Pdf]], [[Cdf]], and [[Covariance]]. I'll demonstrate how all built-in statistical functions can be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index). I'll discuss how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all built-in statistical functions can compute weighted statistics, where each point is assigned a different weight. I'll briefly touch on this feature as a segue into next week's topic, Importance Sampling.

This talk can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-08-21-Statistical-Functions.wmv Statistical-Functions.wmv]. The model built during this talk is available for download at [[media:Intro to Statistical Functions.ana|Intro to Statistical Functions.ana]].

=== [[RankCorrel|Spearman Rank Correlation]] ===

'''Date and Time:''' Thursday, 25 March 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Many measures for quantifying the degree of statistical dependence between quantities are used in statistics. THe two most commonly used are [[Correlation|Pearson's Linear Correlation]] and [[RankCorrel|Spearman's Rank Correlation]], computed respectively in Analytica by the functions [[Correlation]] and [[RankCorrel]]. Pearson's [[Correlation]], which is what people usually mean when they just use the term "Correlation", is a measure of how linear the relationship between two variables is. Spearman's [[RankCorrel|Rank Correlation]] is a measure of monotonic the relationship between two variables is.

This talk provides an introduction to the concept of [[RankCorrel|rank correlation]], how it is distinguished from standard [[Correlation|Pearson correlation]], and what it measures. There are several notable and rather diverse uses of [[RankCorrel]], which include these (and probably many others):
* A quantitative measure of the degree to which two variables are monotonically related. (E.g., the degree to which an increase in one leads to an increase, or decrease, in the other).
* Testing (from measurements) whether two factors are statistically dependent
* Importance analysis: Determining how much the uncertain of an input contributes to the uncertainty of an output.
* Sampling from joint distributions with arbitrary marginals and specified rank-correlations ([[Correlate_With]] and [[Correlate_Dists]])

I will focus mostly on the first two factors in this talk (previous webinars on [[#Sensitivity_Analysis_Topics|Sensitivity Analysis]] have covered the Importance Analysis usage to some extent, and a previous webinar on [[#Correlated_and_Multivariate_Distributions|Correlated and Multivariate Distributions]] has covered the last point).

Standard hypothesis tests exist for determining whether two factors are statistically dependent by testing the hypothesis that their [[RankCorrel|rank correlation]] is non-zero (null hypothesis that it is zero). When the P-value of these tests is less than 5% (or 1%), you would be justified in concluding that the two variables are statistically dependent. I will demonstrate how to compute this P-value.

Then I will introduce a new analysis of [[RankCorrel|rank correlation]] that I came up with, which I think is novel and potentially pretty useful, somewhat related to the classical hypothesis tests just mentioned. Suppose you gather a small sample of data on two variables in a study and you want to determine how strong the monotonicity between the two variables is. You can compute the ''sample rank correlation'' for the data set, but this is only an estimate since you have a small sample size and thus sampling error may throw off this estimate. So suppose we imagine there is some "true" underlying rank correlation between the variables (this in itself is a new concept, which I will make precise). From your data set, you have some knowledge about the true value of this underlying rank correlation -- the larger your sample size, the more precise your knowledge is. The new technique I describe here computes a (posterior) distribution over the true underlying rank correlation, from which you can express your rank correlation result as a range (such as rc=0.6±0.2), and answer questions such as what is the probability that the underlying rank correlation is between -0.1 and 0.1, P(-0.1 < rc < 0.1), or P(rc>0), etc. Although this is essentially a posterior distribution, there is no prior distribution involved or needed to computate it, so it is simply a function of the measured data and of the sample size. It really is a probability distribution on the underlying rank correlation, not just a P-value, making it much more useful.

This new analysis is also useful for quantifying the probability that two factors are independent in a manner not possible with the classical tests. The classical P-value of the aforementioned tests measure the probability of a Type II error for the hypothesis that variables are dependent. These tests do not provide the probability of a Type I error, which would be the criteria for concluding that a claim of statistical independence is statistically signficant. This new measure, however, can justifiably be used for quantifying a claim of statistical independence since it allows P(-c<rc<c) to be computed for any c.

I will demonstrate how this new analysis of rank correlation works and is encoded within Analytica, and show how to read off the interesting results.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2010-03-25-Rank-Correlation.wmv Rank-Correlation.wmv]. The model files created during the talk are available at: [[media:Rank-Correlation-Examples.ana|Rank-Correlation-Examples.ana]] and [[media:Rank-Correlation-Analysis.ana|Rank-Correlation-Analysis.ana]].

=== Statistical Functions in Analytica 4.0 ===

Date and Time: Thursday, Aug 16, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

A statistical function is a function that process a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica 4.0. In Analytica 4.0, all built-in statistical functions can now be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index), eliminating the need for separate function libraries. I will demonstrate this use, as well as several new statistical functions, e.g., [[Pdf]], [[Cdf]], [[Covariance]]. I will explain how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all statistical functions in Analytica 4.0 can compute weighted statistics, where each point is assigned a different weight. I'll cover the basics of sample weighting, and demonstrate some simple examples of using this for computing a Bayesian posterior and for importance sampling from an extreme distribution.

The Analytica model file that had resulted by the end of the presentation can be downloaded here:
[[media:User Group Webinar - Statistical Functions.ANA | User Group Webinar - Statistical Functions.ANA]].

=== The [[Large Sample Library: User Guide|Large Sample Library]] ===

'''Date and Time:''' Thursday, 18 Feb 2010 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Large Sample Library: User Guide|Large Sample Library]] is an Analytica library that lets you run a Monte Carlo simulation for a large model or a large sample size that might otherwise exhaust computer memory, including virtual memory. It breaks up a large sample into a series of batch samples, each small enough to run in memory. For selected variables, known as the ''Large Sample Variables'' or ''LSVs'', it accumulates the batches into a large sample. You can then view the probability distributions for each LSV using the standard methods — confidence bands, PDF, CDF, etc. — with the full precision of the large sample.

Memory is saved by not storing results for non-LSVs.

This presentation introduces this library and how to use it.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2010-02-18-Large-Sample-Library.wmv Large-Sample-Library.wmv]. The Large Sample library can be downloaded for use in your own models from the [[Large Sample Library: User Guide]] page. The two example models used during this webinar were: [[media:Enterprise model3.ANA|Enterprise model3.ana]] and [[media:Simple example for Large Sample Library.ana|Simple example for Large Sample Library.ana]].

= Sensitivity Analysis Topics =

=== Tornado Charts ===

'''Time and Date:''' Thursday, 20 Mar 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract:'''

[[Image:Tornado plot.png]]

A tornado chart depicts the result of a '''local sensitivity analysis''', showing how much a computed result would change if each input were varied one input at a time, with all other inputs held to their baseline value. The result is usually plotted with horizontal bars, sorted with larger bars on top, resulting in a graph resembling the shape of a tornado, hence the name. There a numerous variations on tornado charts, resulting from different ways of varying the inputs, and in some cases, different metrics graphed.

This talk will walk through the steps of setting up a Tornado chart, and explore different variations of varying inputs. We'll also explore some more complex issues that can arise when some inputs are arrays.

The model used during this talk is here: [[media:Tornado Charts.ANA|Tornado Charts.ana]] (the stuff for the talk was in the Tornado Analysis module). You can watch a recording of this webinar from [http://AnalyticaOnline.com/WebinarArchive/2008-03-20-Tornado-Charts.wmv Tornado-Charts.wmv].

=== Advanced Tornado Charts -- when inputs are Array-Valued ===

'''Date and Time:''' Thursday, April 17, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The webinar of 20-Mar-2008 ([http://AnalyticaOnline.com/WebinarArchive/2008-03-20-Tornado-Charts.wmv Tornado-Charts.wmv], see webinar archives) went through the fundamentals of setting up a local sensitivity analysis and plotting the results in the form of a tornado chart. That webinar also discussed the many variations of tornado analyses (or more generally, local sensitivity analyses) that are possible.

This talk builds on those foundations by going a step further and addressing tornado analyses when some of the input variables are array-valued. The presence of array-valued inputs introduces many additional possible variations of analyses, as well as many modeling complications. For example, a local sensitivity analysis varies one input at a time, but that could mean you vary each input variable (as a whole) at a time, or it could mean that you vary each cell of each input array individually. Either is possible, each resulting in a different analysis. Some of these variations compute the correct result automatically through the magic of array abstraction, once you've set up the basic tornado analysis that we covered in the first talk, while other require quite a bit of additional modeling effort. However, even the ones that produce the correct result can often be made more efficient, particularly when the indexes of each input variable are different across input variables.

When we do opt to vary input arrays one cell at a time, the display of the results may be dramatically effected. Although we can keep the results in an array form, the customary tornado chart require us to ''flatten'' the multi-D arrays and label each bar on the chart with a cell coordinate.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-04-17-Tornados-With-Arrays.wmv Tornados-With-Arrays.wmv]. This webinar made use of the following models: [[media:Sales Effectiveness Model with tornado.ana|Sales Effectiveness Model with tornado.ana]], [[media:Biotech R&D Portfolio Tornado.ana|Biotech R&D Portfolio with Tornado.ana]], [[media:Sensitivity Analysis Library.ana|Sensitivity Analysis Library.ana]], and [[media:Sensitivity Functions Examples.ana|Sensitivity Functions Examples.ana]]. See [[The Sensitivity Analysis Library]] for more information on how to use [[media:Sensitivity Analysis Library.ana|Sensitivity Analysis Library.ana]] in your own models.

= Financial Analysis =

=== Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]]) ===

'''Date and Time:''' 18 Dec 2008, 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This is Part 3 of a multi-part webinar series where we have been covering the modeling and evaluation of cash flows over time in an interactive exercise-based webinar format, where concepts are introduced in the form of modeling exercises, and participants are asked to complete the exercises in Analytica during the webinar. Part 3 covers Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]]), and includes seven modeling exercises.

To speed the presentation up, I am providing the exercises in advance: [[media:NPV_and_IRR.ppt|NPV_and_IRR.ppt]]. I urge you to take a shot at completing them before the webinar begins, and we'll advance through the exercises more rapidly so as to complete the topic material within the hour. By attempting the exercises in advance, you'll have a good opportunity to compare your solutions to mine, and to ask questions about things you got stuck on.

A dollar received today is not worth the same as a dollar received next year. Taking this time-value of money (or more generally, time-value of utility) into account is very important when comparing cash flows over time that result from long-term capital budgeting decisions. Net Present Value (NPV) and Internal Rate of Return (IRR) are the two most commonly used metrics examining the effective value of an investment's cash flow over time. Both concepts are pervasive in decision-analytic models.

This webinar will be highly interactive. Fire up a instance of Analtyica as you join on. As I introduce each concept, I'll provide you with cash flow scenarios, and give you a chance to compute the result yourself using Analytica. This talk is intended for people who are not already well-versed in NPV and IRR, or for people who already have a good background with those concepts but are new to Analytica and thus can learn from the interactive practice of addressing these exercises during the talk.

See also the materials from Parts 1 and 2 (Net Present Value, 20 Nov 2008 and 4 Dec 2008) elsewhere on this page. This session begins with the model [[media:Cash Flow Metrics 2.ana|Cash Flow Metrics 2.ana]], and ends with [[media:Cash Flow Metrics 3.ana|Cash Flow Metrics 3.ana]]. You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-12-18-IRR.wmv IRR.wmv].

=== Bond Portfolio Analysis ===

'''Date and Time: ''' 11 Dec 2008, 10:00am Pacific Standard Time

'''Presenter:''' Rob Brown, Incite! Decision Technologies

'''Abstract'''

I demonstrate how to value a bond portfolio in which bonds are bought and sold on an uncertain frequency. The demonstration shows how Intelligent Arrays and related functions can greatly simplify calculations of multiple dimensions that would typically require multiple interconnected sheets in a spreadsheet or nested do-loops in a procedural language.

You can watch a recording of this webinar at
[http://AnalyticaOnline.com/WebinarArchive/2008-12-11-Bond-Portfolio-Analysis.wmv Bond-Portfolio-Analysis.wmv]. The model underlying the presentation is [[media:Bond Portfolio Valuation.ana|Bond Portfolio Valuation.ana]], and the power point slides are at [[media:Bond Portfolio Valuation.ppt|Bond Portfolio Valuation.ppt]].

=== Net Present Value ([[NPV]]) ===

'''Date and Time:''' Part I : Thursday, 20 Nov 2008, 10:00am Pacific Standard Time
::::Part II : Thursday, 4 Dec 2008, 10:00am Pacific Standard Time

(Parts 1 & 2 cover NPV -- part 3, listed now separately, covers IRR)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A dollar received today is not worth the same as a dollar received next year. Taking this time-value of money (or more generally, time-value of utility) into account is very important when comparing cash flows over time that result from long-term capital budgeting decisions. Net Present Value ([[NPV]]) and Internal Rate of Return ([[IRR]]) are the two most commonly used metrics examining the effective value of an investment's cash flow over time. Both concepts are pervasive in decision-analytic models.

This multi-part webinar provides an introduction to the concepts of present value, discount rate, [[NPV]] and [[IRR]]. We'll discuss the interpretation of ''discount rate'', and we'll get practice computing these metrics in Analytica. We'll examine the pitfalls of each metric, and we'll examine the interplay of each metric with explicitly modelled uncertainty (including the concepts of Expected NPV (ENPV) and Expected IRR (EIRR)).

This webinar will be highly interactive. Fire up a instance of Analtyica as you join on. As I introduce each concept, I'll provide you with cash flow scenarios, and give you a chance to compute the result yourself using Analytica. This talk is intended for people who are not already well-versed in [[NPV]] and [[IRR]], or for people who already have a good background with those concepts but are new to Analytica and thus can learn from the interactive practice of addressing these exercises during the talk.

I have assembled quite a bit of material, which I believe will fill two webinar sessions. Part 1 will focus mostly on present value, NPV, discount rate, and the use of NPV with uncertainty. Part 2 will focus mostly on IRR, several "gotchas" with IRR, and MIRR.

Materials:
* [[media:Cash Flow Metrics 2.ana|Cash Flow Metrics 2.ana]] : Model at end of second session
* [[media:Cash Flow Metrics 1.ANA|Cash Flow Metrics 1.ana]] : Model at end of first session
* [http://AnalyticaOnline.com/WebinarArchive/2008-11-20-NPV-and-IRR1.wmv NPV-and-IRR1.wmv] : Webinar recording of Part 1.
* [http://AnalyticaOnline.com/WebinarArchive/2008-12-04-NPV-and-IRR2.wmv NPV-and-IRR2.wmv] : Webinar recording of Part 2.

Note: Part 1 covered 5 exercises, covering present value, discount rate, modeling certain cash flows, computing NPV, and graphing the NPV curve. Part 2 added exercises 6-9, covering cash flows at non-uniformly-spaced time periods, valuating bonds and treasury notes, cash flows with uncertainty, and using the CAPM to find invester-implied corporate discount rate.

The "class" will continue with Part 3 beginning with Internal Rate of Return.

= Data Analysis Techiques =

=== Statistical Functions ===

'''Date and Time:''' Thursday, May 22, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A statistical function is a function that processes a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica 4.0. In Analytica 4.0, all built-in statistical functions can now be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index), eliminating the need for separate function libraries. I will demonstrate this use, as well as several new statistical functions, e.g., [[Pdf]], [[Cdf]], [[Covariance]]. I will explain how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all statistical functions in Analytica 4.0 can compute weighted statistics, where each point is assigned a different weight. I'll cover the basics of sample weighting, and demonstrate some simple examples of using this for computing a Bayesian posterior and for importance sampling from an extreme distribution.

This talk is appropriate for moderate to advanced users.

A recording of this webinar can be watched at
[http://AnalyticaOnline.com/WebinarArchive/2008-05-22-Statistical-Functions.wmv Statistical-Functions.wmv].
The model created during this webinar is at [[media:Statistical Functions.ANA|Statistical Functions.ana]].

=== Principal Components Analysis (PCA) ===

'''Date and Time:''' 15 Jan 2009, 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Principal component analysis (PCA) is a widely used data analysis technique for dimensionality reduction and identification of underlying common factors. This webinar will provide a gentle introduction to PCA and demonstrate how to compute principal components within Analytica. Intended to be at an introductory level, with no prior experience with PCA (or even knowledge of what it is) assumed.

The model developed during this talk, where the principal components were computed for 17 publically traded stocks based on the previous 2 years of price change data is [[media:Principal Component Analysis.ana|Principal Component Analysis.ana]]. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-01-15-PCA.wmv PCA.wmv].

=== Variable Stiffness Cubic Splines ===

'''Date and Time:''' 2 October 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Brian Parsonnet, ICE Energy

'''Abstract'''

The Variable Stiffness Cubic Spline is a highly robust data smoothing and interpolation technique. A stiffness parameter adjusts the variability of the curve. At the extreme of minimal stiffness, the curve approaches a cubic spline (like [[CubicInterp]]) that passes through all data points, while at the other extreme of maximal stiffness, the spline curve becomes the best-fit line. Weight parameters can be used to constrain the curve to include selected points, while smoothing over others. The first, second and third derivatives all exist and are readily available.

I'll introduce and demonstrate [[User-Defined Functions]] that compute the variable stiffness cubic spline and interpolate to new points. I'll also show how these curves can be used to detect or eliminate anomalies in data.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-10-02-Variable-Stiffness-Cubic-Splines.wmv Variable-Stiffness-Cubic-Splines.wmv]. The model and library with the vscs functions will be posted here within a few weeks.

=== Using [[Regression]] ===

'''Date and Time:''' Thursday, May 1, 2008 at 10:00 - 11:00 Pacific Daylight Time

'''Date and Time:''' Thursday, Aug 30, 2007 at 10:00 - 11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Regression analysis is a statistical technique for curve fitting, discovering relationships in data, and testing hypotheses between variables. In this webinar, I will focus on generalized linear regression, which is provided by Analytica's [[Regression]] function, and examine many ways in which is can be used, including fitting simple lines to data, polynomial regression, use of other non-linear terms, and fitting of autoregressive models (e.g., ARMA). I'll examine how we can assess how likely it is the data might have been generated from the particular form of the regression model used. We can also determine the level of uncertainty in our inferred parameter values, and incorporate these uncertainties into a model that uses the result of the regression. The talk will cover Analytica 4.0 functions [[Regression]], [[RegressionDist]], [[RegressionFitProb]], and [[RegressionNoise]].

You can watch a recording of the 1 May 2008 webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-05-01-Regression.wmv Regression.wmv]. The model developed during that webinar is here: [[media:Using_Regression.ANA|Using Regression.ana]]

=== Logistic Regression ===

'''Date and Time:''' Thursday, 5 June 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

''' Abstract'''

''(Features covered in this webinar require Analytica Optimizer)''

Logistic regression is a technique for fitting a model to historical data to predict the probability of an event from a set of independent variables. In this talk, I'll introduce the concept of Logistic regression, explain how it differs from standard [[Regression|linear regression]], and demonstrate how to fit a logistic regression model to data in Analytica. Probit regression is for all practical purposes the same idea as Logistic regression, differing only in the specific functional form for the model. Poisson regression is also similar except is appropriate when predicting a probability distribution over a dependent variable that represents integer "counts". All are examples of generalized linear models, and after reviewing these forms of logistic regression, it should be clear how other generalized linear model forms can be handled within Analytica.

This topic is appropriate for advanced modelers. I will assume familiarity with [[Regression|regression]] (see the earlier talk on the topic), but will not assume a previous knowledge of logistic regression.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2008-06-05-Logistic-Regression.wmv Logistic-Regression.wmv]. The model developed during this webinar can be downloaded from [[media:Logistic_regression_example.ana|Logistic_regression_example.ana]]. You'll also need the file [[media:BreastCancer.data|BreastCancer.data]].

= Bayesian Techniques =

=== Bayesian Posteriors using Importance Sampling ===

'''Date and Time:''' Thursday, September 4, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Several algorithms for computing Bayesian posterior probabilities are special cases of ''importance sampling''. The [[#Importance_Sampling_.28Rare_events.29|webinar of the previous week, Importance Sampling (rare events)]] introduced importance sampling, covered the theory behind it, how it is applied, and how Analytica's sample weighting feature can be use for importance sampling. This webinar continues with importance sampling, this time exploring how it can be used (at least in some cases) to compute Bayesian posterior probabilities.

I'll provide an introduction to what Bayesian posterior probabilities are, describe a couple importance sampling-based approaches to computing them, and implement a few examples in Analytica. Importance sampling techniques for computing posteriors have limited applicability -- in some cases they work well, other not. I'll try to characterize what those conditions are.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-09-04-Posteriors_using_IS.wmv Posteriors_using_IS.wmv]. About two-thirds through the presentation, we noticed a result that seemed to be coming out incorrectly. I explain what the problem was and fix it in [http://AnalyticaOnline.com/WebinarArchive/2008-09-04-Posteriors_using_IS_addendum.wmv Posteriors_using_IS_addendum.wmv]. The models used during this presentation can be downloaded from [[media:Posterior sprinklers.ana|Posterior sprinklers.ana]] and [[media:Likelihood weighting.ana|Likelihood weighting.ana]].

=== Importance Sampling (Rare events) ===

'''Date and Time:''' Thursday, 28 Aug 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Importance sampling is a technique that simulates a target probability distribution of interest by sampling from a different sampling distribution and then re-weighting the sampled points so that computed statistics match those of the target distribution. The technique has has applicability when the target distribution is difficult to sample from directly, but where the probability density function is readily available. The technique produces valid results in the large sample size limit for any selection of sampling distribution (provided it is absolutely continuous with respect to the target distribution), but best results (i.e., fastest convergence with smaller sample size) are obtained when a good sampling distribution is used. The technique is commonly used for rare-event sampling, where you want to ensure greater sampling coverage in the tails of distributions, where few samples would occur with standard Monte Carlo sampling. During the talk, we develop a rare event model. It also has applicability to the computation of Bayesian posteriors, and sampling of complex distribution.

In this talk we cover the theory behind importance sampling and introduce the [[SampleWeighting|sample weighting]] mechanism that is built into Analytica. We develop a rare-event model to demonstrate how the weighting mechanism is used to achieve the importance sampling. Next week we'll continue with an example of computing a Bayesian posterior probability.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-08-28-Importance-Sampling.wmv Importance-Sampling.wmv]. The model developed during this talk can be downloaded from: [[media:Importance sampling rare events.ANA|Importance Sampling rare events.ana]].

= Presenting Models to Others =

=== Guidelines for Model Transparency ===

'''Date and Time:''' 19 Feb 2009, 10:00am Pacific Standard Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

What makes Analytica models easy for others to use and understand? I will review some example models that illustrate ways to improve transparency -- or opacity. Feel free to send me your candidates ahead of time!
We'll review some proposed guidelines. I hope to stimulate a discussion about what you think works well or not, and enlist your help in refining these guidelines.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-02-19-Transparency-Guidelines.wmv Transparency-Guidelines.wmv].

=== Creating Control Panels ===

'''Date and Time:''' Thursday, May 29, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

It is quite easy to put together "control panels" or "forms" for your Analytica models by creating input and output nodes for the inputs and outputs of interest to your model end users. This webinar will cover the basic steps involved in creating and arranging these forms, along with some tricks for making the process efficient. We'll cover the different types of input and output controls that are currently available, the use of text nodes to create visual groupings, use of images and icons, and the alignment commands that make the process very rapid. We'll learn how to change colors, and look at the use of buttons very briefly. This talk is appropriate for beginning Analytica users.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-05-29-Control-Panels.wmv Control-Panels.wmv] (required Windows Media Player). The model used during this webinar is at [[media:Building Control Panels.ana|Building Control Panels.ana]].

=== Sneak preview of Analytica Web Publisher ===

'''Date and Time:''' Thursday, February 21, 2008, 10:00 - 11:00 Pacific Standard Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

In this week's webinar, Max Henrion, Lumina's CEO, will provide a sneak preview of the Analytica Web Publisher. AWP offers a way to make Analytica models easily accessible to anyone with a web browser. Users can open a model, view diagrams and objects, change input variables, and view results as tables and graphs. Users will also be able to save changed models, to revisit them in later sessions. Model builders can upload models into AWP directly from their desktop. Usually, AWP directories are password protected, so only authorized users can view and use models. But, we also plan to make a free AWP directory available for people who want to share their models openly.

AWP is nearing release for alpha testing. We will welcome your comments and hearing how you might envisage using AWP.

''This webinar was not recorded.''

= Application Integration Topics =

=== OLE Linking ===

'''Time and Date:''' Thursday, 27 Mar 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract:'''

OLE linking is a commonly used methods for linking data from Excel spreadsheets into Analytica and results from Analytica into Excel spreadsheets. It can be used with other applications that support OLE-linking as well. The basic usage of OLE linking is pretty simple -- it is a lot like copy and paste. This webinar covers basics of using OLE linking of fixed-sized 1-D or 2-D tables. I also demonstrate the basic tricks you must go through to link index values and multi-D inputs and outputs. In addition, we discuss what some of those OLE-link settings actually do, and explain how OLE-connected applications connect to their data sources.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-03-27-OLE-Linking.wmv 2008-03-27-OLE-Linking.wmv].

Note: Another 10 minute fast-paced video (separate from the webinar) demonstrates linking data from Analytica into Excel, computing something from that data, and linking the result back into Analytica: [http://AnalyticaOnline.com/WebinarArchive/OLE-to-Excel-and-back.wmv OLE-to-Excel-and-back.wmv].

=== Querying an OLAP server ===

'''Date and Time:''' Thursday, February 14, 2008, 10:00 - 11:00 Pacific Standard Time 
(''Note: Schedule change from an earlier posting. This is now back to the usual Thursday time. '')

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

In this session, I'll show how the [[MdxQuery]] function can be used to extract multi-dimensional arrays from an On-Line Analytical Processing (OLAP) server. In particular, during this talk we'll query Microsoft Analysis Services using MDX. In this talk, I'll introduce some basics regarding OLAP and Analysis Services, discuss the differences between multi-dimensional arrays in OLAP and Analytica, cover the basics of the MDX query language, show how to form a connection string for [[MdxQuery]], and import data. I'll also show how hierarchical dimensions can be handled once you get your data to Analytica.

''Note: Use of the features demonstrated in this webinar require the Analytica Enterprise or Optimizer edition, or the Analytica Power Player. They are also available in ADE.''

The model created during this webinar is available here: [[media:Using MdxQuery.ana|Using MdxQuery.ana]]. You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-02-14-MdxQuery.wmv MdxQuery.wmv] (requires Microsoft Media Player)

=== Querying an ODBC relational database ===

Date and Time: Thursday, February 7, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this talk I'll review the basics of querying an external relational ODBC database using [[DbQuery]]. This provides a flexible way to bring in data from SQL Server, Access, Oracle, and mySQL databases, and can also be used to read CSV-text databases and even Excel. In this talk, I will cover the topics of how to configure and specify the data source, the rudimentary basics of using SQL, the use of Analytica's [[DbQuery]], [[DbWrite]], [[DbLabels]] and [[DbTable]] functions.

''Note: Use of the features demonstrated in this webinar require the Analytica Enterprise or Optimizer edition, or the Analytica Power Player. They are available in ADE."

You can grab the model created during this webinar from here: [[media:Querying an ODBC relational database.ANA|Querying an ODBC relational database.ana]]. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-02-07-Using-ODBC-Queries.wmv Using-ODBC-Queries.wmv].

=== Calling External Applications ===

Date and Time: Thursday, Oct 18, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

The [[RunConsoleProcess]] function runs an external program, can exchange data with that program, and can be used to perform a computation or acquire data outside of Analytica, that then can be used within the model. I'll demonstrate how this can be used with a handful of programs, and code written in several programming and scripting languages. I'll demonstrate a user-defined function that retrieves historical stock data from a web site.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-10-18-Calling-External-Applications.wmv Calling-External-Applications.wmv] (Requires Windows Media Player)

Files created or used during this webinar can be downloaded:
* [[media:Regular Expression Matching.ANA|Regular Expression Matching.ana]]
* [[media:RegExp.vbs|RegExp.vbs]]
* [[media:Read Historical Stock Data.ana|Read Historical Stock Data.ana]]
* For plotting to gnuplot, these gnuplot command files were used. (Note: You may have to adjust some file paths within these files, and within the model): [[media:Gnuplot-candlesticks.dat|Gnuplot-candlesticks.dat]], [[media:Gnuplot-3dsurface.dat|Gnuplot-3dsurface.dat]]
* [[media:ReadURL.exe|ReadURL.exe]] (for C++/CLR source code, see [[Retrieving Content From the Web]])

The example of retrieving stock data from Yahoo Finance is also detailed in an article here: [[Retrieving Content From the Web]]

=== New Functions for Reading Directly from an Excel File ===

'''Date and Time:''' Thursday, 24 April 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

'''(Feature covered requires Analytica Enterprise or better)'''

Hidden within the new release of Analytica 4.1 are three new functions for reading values directly from Excel spreadsheets: [[OpenExcelFile]], [[WorksheetCell]], [[WorksheetRange]]. These provide an alternative to OLE linking and ODBC for reading data from spreadsheets, which may be more convenient, flexible and reliable in many situations. We have not yet exposed these functions on the Definitions menu or in the Users Guide in release 4.1, since they are still in an experimental stage. I would like know that they have been "beta-tested" in a variety of scenarios before we fully expose them (also, the symmetric functions for writing don't exist yet). In this webinar, I will introduce and demonstrate these functions, after which you can start using them with your own problems.

The model created during this talk is here: [[Media:Functions for Reading Excel Worksheets.ana|Image:Functions for Reading Excel Worksheets.ana]]. It read from the example that comes with Office 2003, to which we added a few range names during the talk, resulting in [[media:SOLVSAMP.XLS|SolvSamp.xls]]. Place the excel file in the same directory as the model. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-04-24-Reading-From-Excel.wmv Reading-From-Excel.wmv].

=== Reading Data from URLs to a Model ===

'''Date and Time:''' Thursday, 27 Aug 2009, 10:00am-11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''Requires Analytica Enterprise''

The new built-in function, [[ReadFromUrl]], can be used to read data (and images) from websites, such as HTTP web pages, FTP pages, or even web services like SOAP. In this webinar, I'll demonstrate the use of this function in several ways, including reading live stock and stock option price data, posting data to a web form, retrieving a text file from an FTP site, supplying user and password credentials for a web site or ftp service, downloading and displaying images including customized map and terrain images, and querying a SOAP web service.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-08-27-ReadFromUrl.wmv ReadFromUrl.wmv]. The model with the examples shown during the webinar is at [[media:Reading_Data_From_the_Web.ana|Reading_Data_From_the_Web.ana]].

=== Using the Analytica Decision Engine (ADE) from ASP.NET ===

'''Date and Time:''' Thursday, April 10, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Fred Brunton, Lumina Decision Systems

The Analytica Decision Engine (ADE) allows you to utilize a model developed in Analytica as a computational back-end engine from a custom application. In this webinar, we'll create a simple active web server application using ASP.NET that sends inputs submitted by a user to ADE, and displays results computed by ADE on a custom web page. In doing this, you will get a flavor how ADE works and how you program with it. If you've never created an active server page, you may enjoy seeing how that is done as well. This introductory session is oriented more towards people who do not have experience using ADE, so that you can learn a bit more about what ADE is and where it is appropriate by way of example.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-04-10-ADE-from-ASPNET.wmv ASP-from-ASPNET.wmv]. To download the program files that were created during this webinar [[Media:WebSite6.rar| Click here]].

= Optimization =

=== [[Analytica_4.3#Structured_Optimization|Introduction to Structured Optimization]] ===

'''Date and Time:''' Thursday, Febrary 24, 2011 at 10:00am PST (1:00pm EST, 6:00pm GMT)

'''Presenter''': Paul Sanford, Lumina Decision Systems

'''Abstract'''

[[Analytica 4.3]] is now available for beta testing and will be released in early March. The new version includes expanded optimization capabilities and simplified workflow for encoding optimization problems. The new '''Structured Optimization''' framework in 4.3 is centered around a new function, [[DefineOptimization]](), which replaces all three of the previous type-specific functions: LPDefine(), QPDefine() and NLPDefine(). It also introduces a new node type, '''Constraint''', which allows you to specify constraints using ordinary expressions. Paul will build up some basic examples using Structured Optimization and field questions from users.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2011-2-24-Structured-Optimization.wmv Structured-Optimization.wmv]. The example models used during this webinar are: [[Media:Beer Distribution LP1.ana|Beer Distribution LP1.ana]],[[Media:Beer Distribution LP2.ana|Beer Distribution LP2.ana]], [[File:Plane Allocation LP.ana|Plane Allocation LP.ana]],[[File:polynomial NLP.ana|polynomial NLP.ana]]

=== Introduction to Linear and Quadratic Programming ===

Date and Time: Thursday, Oct 11, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

This talk is an introduction to linear programming and quadratic programming, and an introduction to solving LPs and QPs from inside an Analytica model (via Analytica Optimizer). LPs and QPs can be efficiently encoded using the Analytica Optimizer functions [[LpDefine]] and [[QpDefine]]. I'll introduce what a linear program is for the sake of those who are not already familiar, and examine some example problems that fit into this formalism. We'll encode a few in Analytica and compute optimal solutions. Although LPs and QPs are special cases of non-linear programs (NLPs), they are much more efficient and reliable to solve, avoid many of the complications present in non-linear optimization, and fully array abstract. Many problems that initially appear to be non-linear can often be reformulated as an LP or QP. We'll also see how to compute secondary solutions such as dual values (slack variables and reduced prices) and coefficient sensitivies. Finally, [[LpFindIIS]] can be useful for debugging an LP to isolate why there are no feasible solutions.

You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-10-11-LP-QP-Optimization.wmv LP-QP-Optimization.wmv] (requires Windows Media Player)

The model file created during this webinar is here: [[media:LP QP User Group.ANA|LP QP Optimization.ana]]

=== Non-Linear Optimization ===

Date and Time: Thursday, Oct 4, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

This talk focuses on the problem of maximizing or minimizing an objective criteria in the presence of contraints. This problem is referred to as a non-linear program, and the capability to solve problems of this form is provided by the Analytica Optimizer via the [[NlpDefine]] function. In this talk, I'll introduce the use of [[NlpDefine]] for those who have not previously used this function, and demonstrate how NLPs are structured within Analytica models. I'll examine various challenges inherent in non-linear optimization, tricks for diagnosing these and some ways to address these. We'll also examine various ways in which to structure models for parametric analyses (e.g., array abstraction over optimization problems), and optimizations in the presence of uncertainty.

You can watch a recording of this session here: [http://AnalyticaOnline.com/WebinarArchive/2007-10-04-Nonlinear-Optimization.wmv Nonlinear-Optimization.wmv]

During the talk, these two models were created:
* [[media:Simple Nonlinear optimization.ANA|Simple Nonlinear Optimization.ana]]
* [[media:Nonlinear asset allocation.ANA|Nonlinear asset allocation.ana]]

= Vertical Applications and Case Studies =

=== Regional Weather Data Analysis ===

'''Date and Time:''' Thursday, 22 April 2010 10:00am Pacific Daylight Time

'''Presenter:''' Brian Parsonnet, Ice Energy

'''Abstract'''

There are numerous sources of weather data on the web. Users of this data face a few common problems: how to gather the data in volume, how to normalize the data regardless of source, and how to analyze the results to generate insight. Analytica is the perfect tool to address all three issues simply and efficiently. A sample model will be shown illustrating some of techniques.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-04-22-Regional-Weather-Analysis.wmv Regional-Weather-Analysis.wmv]. The model shown during the talk is [[media:Weather analysis.ana|Weather analysis.ana]], and the data file used by this model for Burbank weather can be downloaded from [[media:Burbank.zip|Burbank.zip]] (remember to Unzip it first to Burbank.txt). To avoid issues with ownership of the data, the temperatures in this file have been randomized (so the data is not accurate) and other fields zeroed out, but this will still allow you to play with the model and data.

=== Automated Monitoring and Failure Detection ===

'''Date and Time:''' 5 Feb 2009, 10:00am Pacific Standard Time

'''Presenter:''' Brian Parsonnet, ICE Energy

'''Abstract'''

In many complex physical systems, the automatic and proactive detection of system failures can be highly beneficial. Often dozens of sensor readings are collected over time, and a computer analyzes these to detect when system behavior is deviating from normal. Sounding an alert can then facilitate early intervention, perhaps catching a component that is just starting to go bad.

In a complex physical system with multiple operating modes and placed in a changing environment, anomaly detection is a very difficult problem. Simple sensor thresholds (and other related approaches) lack context-dependence, often making these simple approaches insufficient for the task. What is normal for any given sensor depends on the system's operating mode, time of day, activities in progress, and environmental factors. Simple thresholds that don't take such context into account either end up being so loose that they miss legitimate anomalies, or so tight that too many excess alarms are generated during normal conditions.

In this webinar, I'll show an expert system I've developed in Analytica that detects anomalies and developing failures in our deployed cooling system products. Data from dozens of sensors is collected in 5 minute intervals and the system transitions through multiple operating modes, daily and seasonal environmental fluctuations, and system demands. The Analytica model provides a framework in which complex rules that take multiple factors into account can be expressed, and used to estimate acceptable upper and lower operating ranges that are dynamically adjusted across each moment in time, taking into account whatever context is available. The Analytica environment presents a very readable and understandable language for expressing monitoring rules, and the overall transparency enables us to spot where other rules are needed and what they need to be.

[[Image:AutoMonitoring.png|frame|none|Graph illustrates how upper and lower bounds on operating range is adjusted to context. Actual sensor data is green, the red and blue lines show the computed bounds on acceptable operating range at each point in time.]]

A recording of this webinar can be watched at [http://AnalyticaOnline.com/WebinarArchive/2009-02-05-Failure-detection.wmv Failure-Detection.wmv].

=== Data Center Capacity Planning ===

''Please note that this presentation will be on Wednesday rather than Thursday this week.''

'''Date and Time:''' Wednesday, October 21, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

Data center energy demands are on the rise, creating serious financial as well as infrastructural challenges for data center operators. In 2006, data centers were responsible for a costly 1.5 percent of total U.S. electricity consumption, and national energy consumption by data centers is expected to nearly double by 2011. For data center operators, this means that many data centers are reaching the limits of power capacity for which they were originally designed. In fact, Gartner predicts that 50 percent of data centers will discover they have insufficient power and cooling capacity in 2008.

This week's presentation will provide an overview of [http://www.lumina.com/ADCAPT/Analytica%20Data%20Center%20Capacity%20Planning%20Tool%20Overview.pdf ADCAPT -- the Analytica Data Center Capacity Planning Tool]. For this webinar, the User Group will be joining a presentation that is also being given outside of the Analytica User Group, but I (Lonnie) think is also of interest to the User Group community in that it shows of an example of a re-usable Analytica model, containing several very interesting and novel techniques, applied to a very interesting application area.

Due to technical difficulties, this webinar was not recorded.

=== Modeling the Precision Strike Process ===

'''Date and Time:''' Thursday, October 16, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Henry Neimeier, MITRE

'''Abstract'''

We describe a new paradigm for modeling, and apply it to a simple view of the precision strike attack process against mobile targets. The new modeling paradigm employs analytic approximation techniques that allow rapid model development and execution. These also provide a simple dynamic analytic risk evaluation capability for the first time. The beta distribution is used to summarize a broad range of target dwell and execution time scenarios in compact form. The data processing and command and control processes are modeled as analytic queues.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2008-10-16-Precision-Strike-Process.wmv Precision-Strike-Process.wmv]. Several related papers and materials are also available, including:
* [[media:MILCOM96.pdf|A New Paradigm For Modeling The Precision Strike Process (U)]] by H. Neimeier from MILCOM96
* [[media:Milcom96.ana|Milcom96.ana]] -- the model from the talk and above paper.
* [[media:PhalanxMar.pdf|Analytic Uncertainty Modeling Versus Discrete Event Simulation]] by H. Neimeier, PHALANX March 1996.
* [[media:AnalQueNet.pdf|Analytica Queuing Networks]] by H. Neimeier, Proc. 12th Int'l Conf. Systems Dynamics Soc. 1994.
* [[media:Kuskey CAPE MTR-11.pdf|The Architecture of CAPE Models]] by K.P. Kuskey and S.K. Parker, MITRE Tech. Report.
* [[media:Cmac2pap_2_.pdf|Analytical Modeling in Support of C4ISR Mission Assessment (CMA)]] by F.R. Richards, H.A. Neimeier, W.L. Hamm, and D.L. Alexander, 3rd Int'l Symp. on Command and Control Research and Technology, 1997.
* [[media:INCOSE96.pdf|Analyzing Processes with HANQ]] by H. Neimeier and C. McGowan, MITRE, from INCOSE96.
* Functions for drawing ''radar plots'': [[media:Radarplt.ANA|Radarplt.ana]]
* Power point slides: [[media:Cape.ppt|Cape.ppt]], [[media:PGMrisk.ppt|PGMrisk.ppt]], and [[media:JDEMweb.ppt|JDEMweb.ppt]]

=== Modeling Utility Tariffs in Analytica ===

Presenter: Brian Parsonnet, Ice Energy

Date and Time: Thursday, Nov 8, 2007 at 10:00 - 11:00am Pacific Standard Time

Modeling utility tariffs is a tedious and complicated task. There is no standard approach to how a utility tariff is constructed, and there are 1000’s of tariffs in the U.S. alone. Ice Energy has made numerous passes at finding a “simple” approach to enable tariff vs. product analysis, including writing VB applications, involved Excel spreadsheets, using 3rd party tools, or outsourcing projects to consultants. The difficulty stems from the fact that there is little common structure to tariffs, and efforts to standardize on what structure does exist is confounded by an endless list of exceptions. But using the relatively simple features of Analytica we have created a truly generic model that allows a tariff to be defined and integrated in just a few minutes. The technique is not fancy by Analytica standards, so this in essence demonstrates how Analytic’s novel modeling concept can tackle tough problems.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-08-Tariff-Modeling.wmv 2007-11-08-Tariff-Modeling] (Requires Windows Media Player)

=== Modeling Energy Efficiency in Large Data Centers ===

Date and TimeThursday, Oct 25, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter:Surya Swamy, Lumina Decision Systems

Abstract

The U.S. data center industry is witnessing a tremendous growth period stimulated by increasing demand for data processing and storage. This has resulted in a number of important implications including increased energy costs for business and government, increased emissions from electricity generation, increased strain on the power grid and rising capital costs for data center capacity expansion. In this webinar, Analytica's dynamic modeling capabilities coupled with it's advanced uncertainty capabilities, which offer tremendous support in building cost models for planning and development of energy efficient data centers, will be illustrated. The model enables users to explore future technologies, the performance, costs and efficiencies of which are uncertain and hence to be probabilistically evaluated over time.

You can watch a recording of this presentation at: [http://AnalyticaOnline.com/WebinarArchive/2007-10-25-Data-Center-Model.wmv Data-Center-Model.wmv] (Requires Windows Media Player)

= Graphing =

=== Creating Scatter Plots ===

'''Date and Time:''' Thursday, May 15, 2008 at 10:00 - 11:00am Pacific Daylight
Time

'''Date and Time:''' Thursday, Aug 23, 2007 at 10:00 - 11:00am Pacific Daylight
Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This webinar focuses on utilizing graphing functionality new to Analytica 4.0, and specifically, functionality enabling the creative use of scatter plots. The talk will focus primarily on techniques for simultaneously displaying many quantities on a single 2-D graph. I'll discuss several methods in which multiple data sources (i.e., variable results) can be brought together for display in a single graph, including the use of result comparison, comparison indexes, and external variables. I'll describe the basic new graphing-role / filler-dimension structure for advanced graphing in Analytica 4.0, enabling multiple dimensions to be displayed on the horizontal and vertical axes, or as symbol shape, color, or symbol size, and how all these can be rapidly pivoted to quickly explore the underlying data. I'll discuss how graph settings adapt to changes in pivot or result view (such as Mean, Pdf, Sample views).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-05-15-Scatter-Plots.wmv Scatter-Plots.wmv].

Model used: During this webinar, I started with some example data in the model [[media:Chemical elements orig.ANA|Chemical elements.ana]]. The original file is in the form before graph settings were changed. By the end of the webinar, many graph settings had been altered, and various changes made, resulting in [[media:UG_Webinar_Scatter_Plots.ana|Scatter-Plots.ana]] (during the Aug 23 presentation, this was the final model: [[media:Chemical elements2.ANA|Chemical elements2.ana]]).

=== Graph Style Templates ===

''' Date and Time:''' Thursday, February 28, 2008, 10:00 - 11:00 Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Graph style templates provide a convenient and versitile way to bundle graph setup options so that they can be reused when viewing other result graphs. For example, if you've discovered a set of colors and fonts and a layout that creates the perfect pizzazz for your results, you can bundle that into a template where you can quickly select it for any graph. In this talk, I'll introduce how templates can be used and how you can create and re-use your own. I'll show the basics of using existing templates, previewing what templates will look like, and applying a given template to a single result or to your entire model. We'll also see how to create your own templates, and in the process I'll discuss what settings can be controlled from within a template. I'll discuss how graph setup options are a combination of global settings, template settings, and graph-specific overrides. I'll show how to place templates into libraries (thus allowing you to have template libraries that can be readily re-used in different models), and even show how to control a few settings using templates that aren't selectable from the Graph Setup UI. I'll also touch on how different graph setting are associated with different aspects of a graph, ultimately determining how the graph adapts to changes in uncertainty view or pivots.

The model created during this webinar is here: [[media:Graph style templates.ana|Graph style templates.ana]].
You can watch a recording of the webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-02-28-Graph-Style-Templates.wmv Graph-Style-Templates.wmv].

= Scripting =

=== Button Scripting ===

Date and Time: Thursday, Sept. 6, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Max Henrion, Lumina Decision Systems

Abstract

This webinar is an introduction to Analytica's typescript and button scripting. Unlike variable definitions, button scripts can have side-effects, and this can be useful in many circumstances. I'll cover the syntax of typescript (and button scripts), and how scripts can be used from buttons, picture nodes or choice inputs. I'll introduce some of the Analytica scripting language to those who may have seen or used it before. And we'll examine some ways in which button scripting can be used.

You can watch the recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-06-Button-Scripting.wmv Button Scripting.wmv] (Requires Windows Media Player or equiv). The model files and libraries used during the webinar are in [[media:Ana_tech_webinar_on_scripting.zip|Ana_tech_webinar_on_scripting.zip]].

= Analytica User Community =

=== The Analytica Wiki, and How to Contribute ===

'''Date and Time:''' (tentative) Thursday, October 30, 2008, Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The Analytica Wiki is a central repository of resources for active Analytica users. What's more, you -- as an active Analytica user -- can contribute to it. As an Analytica community, we have a lot to learn from each other, and the Analytica Wiki provides one very nice forum for doing so. You can contribute example models and libraries, hints and tricks, and descriptions of new techniques. You can fix errors in the Wiki documentation if you spot them, or add to the information that is there when you find subtleties that are not fully described. If you spend a lot of time debugging a problem, after solving it you could document the issue and how it was solved for your own benefit in the future, as well as for others in the user community who may encounter the same problem. When you publish a relevant paper, I hope you will add it to the page listing publications that utilize Analytica models.

I will provide a quick tour of the Analytica Wiki as it exists today. I'll then provide a tutorial on contributing to the Wiki -- e.g., the basics of how to edit or add content. The [http://Wikipedia.org Wikipedia] has had tremendous success with this community content contribution model, and I hope that after this introduction many of you will feel more comfortable contributing to the Wiki as you make use of it.

Due to a problem with the audio on the recording, the recording of this webinar is not available.

= Licensing or Installation =

=== Reprise License Manager Tutorial ===

'''Date and Time:''' Wednesday, 11 March 2010, 10:00am Pacific Standard Time

'''Presenter:''' Bob Mearns, Reprise Software Inc.

'''Abstract'''

The Reprise License Manager (RLM) allows all Analytica and ADE licenses within an organization to be managed from a central server. RLM can be used with either ''floating'' or ''named-user'' licenses.

This tutorial on RLM administration is being given by Bob Mearns, lead software developer at Reprise Software, Inc., who has over 15 years' experience developing and supporting software license managers. This session will focus on:

* Basic RLM Server Setup
* How and where RLM looks for licenses
* Using the RLM Web Server Admin Interface
* Using RLM diagnostics, new in RLM v8
* A systematic approach to diagnosing license server connectivity problems

There is a big focus in this talk on how to debug problems with the RLM license manager, and in the process many of the technical details pertaining to the RLM setup are covered. This talk is most relevant for IT managers who administer the license server, and for people who may be installing the RLM server who would like a more thorough understanding of how things work. The RLM license manager is used to host centrally managed licenses, which includes floating and named-user licenses.

This talk is being provided by Reprise Software.

This webinar may be viewed here: [http://analyticaonline.com/WebinarArchive/2010-03-11-RLM-troubleshooting.wmv RLM-troubleshooting.wmv]. The trouble-shooting tips document covered in the talk is at [http://reprisesoftware.com/RLM_Troubleshooting_Tips.pdf RLM Troubleshooting Tips].

See also:
* [[Configuring an RLM Server]] -- step-by-step for installing the RLM server
* [[How to Install Analytica 4.2 -- Centrally Managed License]] -- for the Analytica user's side installation

File:Polynomial NLP.ana

2011-02-25T03:43:20Z

Psanford: NLP demonstrating local optima and the influence of initial guesses

NLP demonstrating local optima and the influence of initial guesses

File:Plane Allocation LP.ana

2011-02-25T03:42:12Z

Psanford: Allocates planes to routes to maximize profit

Allocates planes to routes to maximize profit

Tutorial videos

2011-02-25T03:41:05Z

Psanford: /* Introduction to Structured Optimization */

New: I've attempted to impose some categorization on the past user group webinar topics.

The most recent talks were:
* [[#Introduction_to_Structured_Optimization|Introduction to Structured Optimization]] (24 Feb 2011)
* [[#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]] (Session 1: 29 Apr 2010, Session 2:6 May 2010, Session 3:13 May 2010, Session 4:20 May 2010, Session 5:3 June 2010), Session 6:10 June 2010, Session 7:24 June 2010, Session 8:15 July 2010.
* [[#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[#Spearman_Rank_Correlation|Spearman Rank Correlation]] (25 Mar 2010)

= Table and Array Topics =

=== The Basics of Analytica Arrays and Indexes ===

''This webinar is continued across two sessions.''

Date and Time (Part 1): January 10, 2008, 10:00 - 11:00 Pacific Standard Time

Date and Time (Part 1, repeat): January 11, 2008, 10:00 - 11:00 Pacific Standard Time

Date and Time (Part 2): January 17, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

This introductory talk introduces the basic concepts of Analytica indexes and multi-dimensional arrays, as well as the basics of Intelligent Array Abstraction. There are several important differences between Analytica arrays compared to multi-dimensional arrays found in other modeling, database, and programming environments. For example, each dimension of an array is associated with an index object, and there is no inherent ordering to the dimensions of a multi-D array. Intelligent array abstraction is perhaps the most powerful feature in Analytica. The session will include a brief description of what array abstraction does, and how you should take advantage of it.

Part 1 focuses on indexes, 1-D arrays and the uses of the [[Subscript/Slice Operator]].

Part 2 focuses on [[:Category:Array Functions|array functions]], multi-D arrays, and the principles and philosphy of arrays in Analytica.

This talk is intended for beginning Analytica modelers, and for people who have been using Analytica without making substantial use of its array features.

A recording of the two sessions can be viewed at (requires Windows Media Player):
* [http://AnalyticaOnline.com/WebinarArchive/2008-01-11-Intro-to-arrays.wmv Intro-to-arrays (Part 1).wmv]
* [http://AnalyticaOnline.com/WebinarArchive/2008-01-17-Intro-to-arrays2.wmv Intro-to-arrays (Part 2).wmv]

An Analytica model containing the examples created during the webinar can be downloaded from [[media:Intro to intelligent arrays.ana|Intro to intelligent arrays.ana]]. During part 1, the [[media:Plane catching decision with EVIU.ana|Plane catching decision with EVIU.ana]] was also used briefly during the webinar.

=== Manipulating Indexes and Arrays in Analytica Expressions ===

Date and Time: Thursday, Aug 9, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this webinar, I will review many of the common operations applied to indexes and arrays from within Analytica expressions, with a particular emphasis on enhancements in this area that are new to Analytica 4.0. I'll review the often used and very powerful [[Subscript]] and [[Slice]] operations, along with the [[Associative_vs._Positional_Indexing|duality of associational and positional indexing]]. I'll introduce newly introduced extensions for positional indexes, such as the @I, A[@I=n], and @[I=n] operations, and extensions that expose positional duals to various previously-existing associational array functions. I will describe the distinction between index and value contexts in Analytica expressions, along with the distinction between a variable's index value, mid value and sample value, how these may differ ([[Self-Indexed Arrays]]), and how we may access each context-value explicitly. I will also introduce slice assignment -- the ability to assign values to individual slices of an array within an algorithm.

The content of this webinar is most appropriate for users with moderate to advanced Analyica model-building experience.

Here is the Analytica model that was created during this talk: [[media:Indexes and Arrays UG2.ANA | "Indexes and Arrays UG2.ANA"]]. (This wouldn't be very interesting for someone who didn't attend, but it contains the examples we tried).

=== Local Variables ===

'''Date and Time:''' Thursday, 23 July 2009, 10:00-11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

I'll explain distinctions between different types of local variables that can be used within expressions. These distinctions are of primary interest for people implementing [[Meta-Inference]] algorithms, since they have a lot to do with how [[Handle]]s are treated. Analytica 4.2 introduces some new distinctions to the types of local variables, designed to make the behavior of local variables cleaner and more understandable. One type of local variable is the [[LocalAlias]], in which the local variable identifier serves as an alias to another existing object. In contrast, there is the [[MetaVar]], which may hold a [[Handle]] to another object, but does not act as an alias. The only local variable option that existed previously, declared using [[Var..Do]], is a hybrid of these two, which leads to confusion when manipulating [[handle]]s. Since [[LocalAlias..Do]] and [[MetaVar..Do]] have very clean semantics, the use of these when writing [[Meta-Inference]] algorithm should help to reduce that confusion considerably. Inside a [[User-Defined Function]], parameters are also instances of local variables, and depending on how they are declared, may behave as a [[MetaVar]] or [[LocalAlias]], so I'll discuss how these fit into the picture, as well as local indexes and local indexes.

This is appropriate for advanced Analytica modelers.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-07-23-Local-Variables.wmv Local-Variables.wmv]. The analytica file from the webinar is at [[media:Local Variables.ana|Local Variables.ana]], where I've also implemented the exercises that I had suggested at the end of the webinar, so you can look in the model for the solutions.

=== Array Concatenation ===

'''Date and Time:''' Thursday, 25 June 2009 10:00am-11:00 Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Array concatenation combines two (or more) arrays by joining them side-by-side, creating an array having all the elements of both arrays. The special case of list-concatenation joins 1-D arrays or lists to create a list of elements that can function as an index. Array concatenation is a basic, and common, form of array manipulation.

The [[Concat]] function has been improved in Analytica 4.2, so that array concatenation is quite a bit easier in many cases, and the [[ConcatRows]] function is now built-in (formerly it was available as a library function).

I'll take you through examples of array concatenation, including cases that have been simplified with the 4.2 enhancements, to help develop your skills at using [[Concat]] and [[ConcatRows]].

This webinar is appropriate for all levels of Analytica modelers.

You can view a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-06-25-Array-Concatenation.wmv Array_Concatenation.wmv]. The model file created during the webinar is: [[media:Array_Concatenation.ana|Array_Concatenation.ana]].

=== Flattening and Unflattening of Arrays ===

Date and Time: January 31, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

On occassion you may need to flatten a multi-dimensional array into a 2-D table. The table could be called a ''relational representation'' of the data. In some circles it is also refered to as a ''fact table''. Or, you may need to convert in the other direction -- expanding, or unflattening a relational/fact table into a multi-dimensional array. In Analytica, the [[MdTable]] and [[MdArrayToTable]] functions are the primary tools for unflattening and flattening. In this session, I'll introduce these functions and how to use them, several examples, and many variations.

The model developed during this talk is at [[media:Flattening_and_Unflatting_Arrays.ana | Flattening_and_Unflatting_Arrays.ana]]. A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-01-31-Array-Flattening.wmv Array-Flattening.wmv]

=== The [[Aggregate]] Function ===

'''Date and Time:''' Thursday, 2 July 2009, 10:00am - 11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Aggregation is the process of transforming an array based on a fine-grain index into a smaller array based on a coarser-grain index. For example, you might map a daily cash stream into monthly revenue (i.e., reindexing from days to months).

This has always been a pretty common operation in Analytica models, with a variety of techniques for accomplishing it, but it has just become more convenient with the [[Aggregate]] function, new to Analytica 4.2.

In the webinar, I'll be demonstrating the use and generality of the [[Aggregate]] function. In the process, it will also be a chance to review a number of other basic intelligent array concepts, including array abstraction, subscripting, re-indexing, etc.

This webinar is appropriate for all levels of Analytica modelers.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-07-02-Aggregate.wmv Aggregate.wmv]. The model file created during this webinar is: [[media:Aggregate_Function.ana|Aggregate Function.ana]].

=== Sorting ===

'''Date and Time:''' Thursday, 6 Aug 2009, 10:00am-11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This webinar will demonstrate the functions in Analytica that are used to sort (i.e. re-order) data -- the functions [[sortIndex]], [[Rank]], and the new to 4.2 [[Sort]]. I'll cover the basics of using these functions, including how they interact with indexes, how to apply them to arrays of data, and their use with array abstraction. I'll then introduce several new 4.2 extensions for handling multi-key sorts, descending options, and case insensitivity.

This webinar is appropriate for all levels of Analytica modelers.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-08-06-Sorting.wmv Sorting.wmv]. The model file created during the webinar is at [[media:Sorting.ana|Sorting.ana]].

=== [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]] ===

Date and Time: January 24, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

Every dimension of an Analytica array is associated with an index object. [[Array Abstraction]] recognizes when two arrays passed as parametes to an operator or function contain the same indexes. These indexes are more commonly defined by a global index object, i.e., an index object that appears on a diagram as a parallelogram node. However, variable and decision nodes can serve as indexes, and can even have a multi-dimensional value in addition to being an index itself. This is refered to as a [[Self-Indexed Arrays|self index]]. If a variable identifier is used in an expression, the context in which it appears always makes it clear whether the identifier is being used as an index, or as a variable with a value. Self-indexes can arise in several ways, which I will cover. In rare cases, when writing an expression, you may need to be aware of whether you intend to use the index value or the context value of a self-indexed variable. I'll discuss these cases, for example in [[For..Do]] loops, and the use of the [[IndexValue]] function.

In some cases, lists may be used in expressions, and when combined with other results, lists can end up serving as an [[Implicit Dimensions|implicit dimension]] of an array. An implicit dimension is a bit different from a full-fledged index since it has not name, and hence no way to refer to it in an expression where an index parameter is expected. Yet most built-in Analytica functions can still be employed to operate over an implicit index. When an implicit index reaches the top level of an expression, it is promoted to be a self-index. I will explain and demonstrate these concepts.

The model developed during this talk is at [[media:Self-Indexes_Lists_and_Implicit_dimensions.ana|Self-Indexes_Lists_and_Implicit_dimensions.ana]]. A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-01-24-Self-Indexes-Implicit-Dims.wmv Self-Indexes-Implicit-Dims.wmv]

=== Introduction to [[DetermTable]]s ===

'''Date and Time:''' 18 September 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A [[DetermTable]] provides an input view like that of an [[Table|edit table]], allowing you to specify values or expressions in each cell for all index combinations; however, unlike a [[Table|table]], the evaluation of a [[DetermTable|determtable]] conditionally returns only selected values from the table. It is called a determtable because it acts as a deterministic function of one or more discrete-valued variables.
You can conceptualize a determtable as a multi-dimensional generalization of a select-case statement found in many programming languages, or as a value that varies with the path down a decision tree.

[[DetermTable]]s can be used to encode a table of utilities (or costs) for each outcome in a probabilistic model. In this usage, they combine very naturally with [[ProbTable]]s (probability tables) for discrete probabilistic models. They are also extremely useful in combination with [[Choice]] pulldowns, allowing you to keep lots of data in your model, but using only a selected part of that for your analysis. This leads to [[Selective Parametric Analysis]], which is often an effective way of coping with memory capacity limitation in high dimensional models.

In this talk, I'll introduce the [[DetermTable]], show how you create one and describe the requirements for the table indexes. The actual "selection" of slices occurs in the table indexes. Not all indexes have to be selectors, but I'll explain the difference and how the domain attribute is used to establish the table index, while the value is used to select the slice. When you define the domain of a variable that will serve as a [[DetermTable]] index, you have the option of defining the domain as an ''index domain''. This can be extremely useful in combination with a [[DetermTable]], so I will cover that feature as well. It is helpful to understand how the functionality a [[DetermTable]] can be replicated using two nodes -- the first containing an [[Table|Edit Table]] and the second using [[Subscript]]. Despite this equivalence, [[DetermTable]] can be especially convenient, both because it simplifies things by requiring one less node, but also because an [[Table|Edit Table]] can be easily converted into a [[DetermTable]].

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-09-18-DetermTables.wmv DetermTables.wmv]. The examples created while demonstrating the mechanics of [[DetermTable]]s is saved here: [[media:DetermTable intro.ana|DetermTable intro.ana]]. Other example models used were the ''2-branch party problem.ana'' and the ''Compression post load calculator.ana'', both distributed in the Example models folder with Analytica, and the [[media:Loan_policy_selection.ANA|Loan policy selection.ana]] model.

=== [[Table Splicing]] ===

'''Date and Time''': Thursday, August 14, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Edit tables, probability tables and determ tables automatically adjust when their index's values are altered. When new elements are inserted into an index, rows (or columns or slices) are automatically inserted, and when elements are deleted, rows (or columns or slices) are deleted from the tables. This process of adjusting tables is referred to as ''[[Table Splicing|splicing]]''.

Some indexes in Analytica may be computed, so that changes to some input variables could result in dramatic changes to the index value, both in terms of the elements that appear and the order of the elements in the index. This creates a correspondence problem for Analytica -- how do the rows after the change correspond to the rows before the change. Analytica can utilize three different methods for determining the correspondence: associative, positional, or flexible correspondence. I'll discuss what these are and show you how you can control which method is used for each index.

When slices (rows or columns) are inserted in a table, Analytica will usually insert 0 (zero) as the default value for the new cells. It is possible, however, to explicit set a default value, and even to set a different default for each column of the table. Doing so requires some typescripting, but I'll take you through the steps.

Using blank cells as a default value, rather than zero, has some advantages. It becomes quickly apparent which cells need to be filled in after index items are inserted, and Analytica will issue a warning message if blank cells exist that you haven't yet filled in. I'll take you through the steps of enabling blank cells by default.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-08-14-Edit-Table-Splicing.wmv Edit-Table-Splicing.wmv]. (Note: There is a gap in the recording's audio from 18:43-27:35).

=== [[StepInterp|Step Interpolation]] ===

'''Date and Time:''' Thursday, 8 April 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[StepInterp]] function is useful in a number of scenarios, including:
* Discretizing a continuous quantity into a set of finite buckets.
* Looking up a value from a "schedule table" (e.g., tax-rate table, depreciation table)
* Mapping from a date to its fiscal year, when the fiscal year starts on an arbitrary mid-year date.
* Mapping from a cumulated value back to the index element/position.
* Performing a "nearest" or "robust" [[Subscript]] or [[Slice]] operation.
* Interpolating value for a relationship that changes in discrete steps

In this webinar, I'll demonstrate how to use the [[StepInterp]] function on several simple examples.

This webinar is appropriate for beginning Analytica modelers and up.

You can watch a recording of this webinar at: [http://analyticaonline.com/WebinarArchive/2010-04-08-StepInterp.wmv StepInterp.wmv]. You can download the model created during this webinar from [[media:Step Interp Intro.ana|Step Interp Intro.ana]].

=== Analytica Web Player ===

'''Date and Time:''' Thursday, August 7, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

The Analytica Web Player (AWP) is scheduled to launch on July 31, 2008. AWP is a subscription service hosted on Lumina's servers. As a subscriber, you can upload your models to the server and send your colleagues a URL so that they can view your model. To view your models, they need only a Flash-enabled web browser. They can browser your model, change inputs, and evaluate results, all from within their web browser.

In this talk we'll cover the available subscription plans, pricing, limitations, and how you sign up. We'll also demonstrate the process of uploading models and sharing these with colleagues.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-08-07-AWP.wmv AWP.wmv].

=== [[SubTable|SubTables]] ===

'''Date and Time:''' Thursday, 31 July 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[SubTable]] function allows a subset of another edit table to be edited by the user as a different view. To the user, it appears as if he is editing any other edit table; however, the changes are stored in the original edit table. The rows and columns can be transformed to other dimensions in the Subtable, with different index element orders, based on [[Subset]] indexes, and with different number formats.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-31-SubTables.wmv SubTables.wmv]. The model file from this webinar is at [[media:SubTable_webinar.ana]].

=== Edit Table Enhancements in Analytica 4.0 ===

Date and Time: Thursday, Aug 2, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this webinar, I will demonstrate several new edit table functionalities in Analytica 4.0, including:
* Insert [[Choice]] drop-down controls in table cells.
* [[Table Splicing|Splicing]] tables based on computed indexes.
* Customizing the [[TableCellDefault|default cell value(s)]].
* Blank cells to catch entries that need to be filled in.
* [[SubTable]]s
* Using different number formats for each column.

This talk is oriented for model builders with Analytica model-building experience.

The Analytica session that existed by the end of the talk is stored in the following model file: [[Media:Edit_Table_Features.ANA | "Edit Table Features.ana"]].

= Modeling Time =

=== Manipulating Dates in Analytica ===

Date and Time: Thursday, Sept. 13, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this talk, I'll cover numerous aspects relating to the manipulation of dates in Analytica. I'll introduce the encoding of dates as integers and the date origin preference. I'll review how to configure input variables, edit tables, or even individual columns of edit tables to accept (and parse) dates as input. I'll cover date number format capabilities in depth, including how to create your own custom date formats, understanding how date formats interact with your computer's regional settings, and how to restrict a date format to a single column only. We'll also see how axis scaling in graphs is date-aware.

Next, we'll examine various ways to manipulate dates in Analytica expressions. This includes use of the new and powerful functions [[MakeDate]], [[DatePart]], and [[DateAdd]], and some interesting ways in which these can be used, for example, to define date sequences. Finally, we'll practice our array mastery by aggregating results to and from different date granularities, such aggregating from a month sequence to a years, or interpolating from years to months.

The model file resulting by the end of the session is available here: [[media:Manipulating Dates in Analytica.ana|Manipulating Dates in Analytica.ana]].

You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-13-Manipulating-Dates.wmv Manipulating Dates.wmv] (Windows Media Player required) Unfortunately, this one seems to have recorded poorly -- the video size is too small. If you magnify it in your media player, it does become readable. Sorry -- I don't know why it recorded like this.

=== The [[Dynamic]] Function ===

'''Date and Time:''' Thursday, 12 June 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function is used for modeling or simulating changes over time, in which values of variables at time t depend on the values of those variables at earlier time points. Analytica provides a special system index named Time that can be used like any other index, but which also has the additional property that it is used by the [[Dynamic]] function for dynamic simulation.

This webinar is a brief introduction to the use of the Dynamic function and to the creation of dynamic models. I'll cover the basic syntax of the [[Dynamic]] function, as well as various ways in which you can refer to values at earlier time points within an expression. Dynamic models result in influence diagrams that have directed cycles (i.e., where you can start at a node, follow the arrows forward and return to where you started), called dynamic loops. Similar ''cyclic dependencies'' are disallowed in non-dynamic influence diagrams.

During the webinar, we'll loop at several simple examples of Dynamic, oriented especially for those of you with little or no experience with using [[Dynamic]] in models. I'll provide some helpful hints for keeping things straight when building dynamic models. For the more seasoned modelers, I'll also try to fold in a few more detailed tidbits, such as some explanation about how dynamic loops are evaluated, and how variable identifiers are interpreted somewhat differently from within dynamic loops.

The model developed (extension of Fibonacci's rabbit growth model) can be downloaded here: [[media:The Dynamic Function.ana|The Dynamic Function.ana]]. A recording of the webinar can be viewed at
[http://AnalyticaOnline.com/WebinarArchive/2008-06-12-Dynamic-Function.wmv Dynamic-Function.wmv].

=== Modeling Markov Processes in Analytica ===

Date and Time: Thursday, Sept. 20, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Matthew Bingham, Principal Economist, Veritas Economic Consulting

Abstract

The class of mathematical processes characterized by dynamic dependencies between successive random variables is called Markov chains. The rich behavior and wide applicability of Markov chains make them important in a variety of applied mathematical applications including population and demographics, health outcomes, marketing, genetics, and renewable resources. Analytica’s dynamic modeling capabilities, robust array handling, and flexible uncertainty capabilities support sophisticated Markov modeling. In this webinar, a Markov modeling application is demonstrated. The model develops age-structured population simulations using a Leslie matrix structure and dynamic simulation in Analytical.

A recording of this session can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2007-09-20-Markov-Processes.wmv Markov-Processes.wmv] (requires Windows Media Player)

An article about the model presented here: [[media:AnalyticaMarkovtext.pdf|AnalyticaMarkovtext.pdf]]

= Analytica Language Features =

=== [[Local Indexes]] ===

Date and Time: Thursday, Dec. 13, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

A [[Local Indexes|local index]] is an index object created during the evaluation of an expression using either the [[Index..Do]] or [[MetaIndex..Do]] construction. Local indexes may exist only temporarily, being reclaimed when they are no longer used, or they may live on after the evaluation of the expression has completed, as an index of the result. Some operations require the use of local indexes, or otherwise could not be expressed.

In this talk, I'll introduce simple uses of local indexes, covering how they are declared using [[Index..Do]], with several examples. We'll see how to access a local index using the [[Dot operator::A.I|A.I]] operator. I'll discuss the distinctions between local indexes and local variables. I'll show how the name of a local index can be computed dynamically, and I'll briefly cover the [[IndexNames]] and [[IndexesOf]] functions.

The model created during this talk is here: [[media:Webinar_Local_Indexes.ana|Webinar_Local_Indexes.ana]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-12-13-Local-Indexes.wmv Local-Indexes.wmv] (Requires Windows Media Player)

=== [[Handle]]s and [[Meta-Inference]] ===

Date and Time: Thursday, Dec. 6, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

Meta-inference refers to computations that reason about your model itself, or that actually alter your model. For example, if you were to write an expression that counted how many variables are in your model, you would be reasoning about your model. Other examples of meta inference include changing visual appearance of nodes to communicate some property, re-arranging nodes, finding objects with given properties, or even creating a transformed model based on portion of your model's structure.

The ability to implement meta-inferential algorithms in Analytica has been greatly enhanced in Analytica 4.0. The key to implementation of meta-inference is the manipulation of [[Handle]]s to objects (formerly refered to as ''varTerms''). This webinar will provide a very brief introduction to handles and using them from within expressions. I will assume you are pretty familiar with creating models and writing expressions in Analyica, but I will not assume that have previous seen or used Handles. This topic is oriented towards more advanced Analytica users.

The model used/created during this webinar as at: [[media:Handle and MetaInference Webinar.ANA|Handle and MetaInference Webinar.ANA]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-12-06-Handles.wmv Handles.wmv] (Requires Windows Media Player)

=== The [[Iterate]] Function ===

Date and Time: Thursday, Nov. 29, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

With [[Iterate]], you can create a recurrent loop around a large model, which can be useful for iterating until a convergence condition is reached, for example. For complex iterations, where many variables are being updated at each iteration, requires you to structure your model appropriate, bundling and unbundling values within the single iterative loop. With some work, [[Iterate]] can be used to simulate the functionality [[Dynamic]], and thus provides one option when a second [[Time]]-like index is needed (although not nearly as convenient as [[Dynamic]]).

In this session, we'll explore how [[Iterate]] can be used.

Here is the model file developed during the webinar: [[media:Iterate Demonstration.ANA|Iterate Demonstration.ANA]]

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-29-Iterate.wmv Iterate.wmv] (Requires Windows Media Player)

=== The [[Using_References|Reference and Dereference Operators]] ===

Date and Time: Thursday, Nov. 15, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract
The reference operators make it possible to represent complex data structures like trees or non-rectangular arrays, bundle heterogenous data into records, maintain arrays of local indexes, and seize control of array abstraction in a variety of scenarios. Using a reference, an array can be made to look like an atomic element to array abstraction, so that arrays of differing dimensionality can be bundled into a single array without an explosion of dimensions. The flexibilities afforded by references are generally for the advanced modeler or programmer, but once mastered, they come in useful fairly often.

Here is the model used during the webinar: [[media:Webinar Reference and Dereference Operators.ANA|Reference and Dereference Operators.ana]].
Near the end of the webinar, I encountered a glitch that I was not able to resolve until after the webinar was over. This has been fixed in the attached model. For an explanation of what was occurring, see:
[[Analytica_User_Group/Reference_Webinar_Glitch]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-15-Reference-And-Dereference.wmv Reference-And-Dereference.wmv] (Requires Windows Media Player)

=== Writing [[User-Defined Functions]] ===

Date and Time: Thursday, Sept. 27, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

When you need a specialized function that is not already built into Analytica, never fear -- you can create your own [[User-Defined Functions|User-Defined Function (UDF)]]. Creating UDFs in Analytica is very easy. I'll introduce this convenient capability, and demonstrate how UDFs can be organized into libraries and re-used in other models. I'll also review the libraries of functions that come with Analytica, providing dozens of additional functions.

After this introduction to the basics of UDFs, I'll dive into an in-depth look at [[Function Parameter Qualifiers]]. There is a deep richness to function parameter qualifiers, mastery of which can be used to great benefit. One of the main objectives for a UDF author, and certainly a hallmark of good modeling style, should be to ensure that the function fully array abstracts. Although this usually comes for free with simple algorithms, it is sometimes necessary to worry about this explicitly. I will demonstrate how this objective can often be achieved through appropriate function parameter qualification.

Finally, I will cover how to write a custom distribution function, and how to ensure it works with [[Mid]], [[Sample]] and [[Random]].

This talk is appropriate for Analytica modelers from beginning through expert level. At least some experience building Analytica models and writing Analytica expressions is assumed.

The model created during this webinar, complete with the UDFs written during that webinar, can be downloaded here: [[media:Writing User Defined Functions.ana|Writing User Defined Functions.ana]].

You can watch this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-27-Writing-UDFs.wmv Writing-UDFs.wmv] (Windows Media Player required)

=== Custom Distribution Functions ===

'''Date and Time:''' Thursday, 24 July 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Analytica comes with most of the commonly seen [[:category:Distribution Functions|distributions]] built-in, and many additions [[:category:Distribution Functions|distribution functions]] available in the standard libraries. However, in specific application areas, you may encounter distribution types that aren't already provided, or you may wish to create a variation on an existing distribution based on a different set of parameters. In these cases, you can create your own ''User-Defined Distribution Function'' (UDDF). Once you've created your function, you can utilize it within your model like you would any other distribution function.

User-defined distribution functions are really just instances of [[User-Defined Functions]] (UDFs) that behave in certain special ways. This webinar discusses the various functionalities that a user-defined distribution function should exhibit and various related considerations. Most fundamentally, the defining feature of a UDDF is that it returns a median value when evaluated in Mid mode, but a sample indexed by Run when evaluated from Sample mode. This contrasts with non-distribution functions whose behavior does not depend on the Mid/Sample evaluation mode. Custom distributions are most often implemented in terms of existing distributions (which includes Inverse CDF methods for implementing distributions), so that this property is achieved automatically since the existing distributions already have this property. But in less common cases, UDDFs may treat the two evaluation modes differently.

When you create a UDDF, you may also want to ensure that it works with [[Random]]() to generate a single random variate, and supports the [[Distribution Functions#The Over parameter|Over parameter]] for generating independent distributions. You may also want to create a companion function for computing the density (or probability for discrete distributions) at a point, which may be useful in a number of contexts including, for example, during importance sampling. I'll show you how these features are obtained.

There are several techniques that are often used to implement distribution functions. The two most common, especially in Analytica, are the ''Inverse CDF'' technique and the ''transformation from existing distributions'' method. I'll explain and show examples of both of these. The Inverse CDF is particularly convenient in that it supports all sampling methods (Median Latin Hypercube, Random Latin Hypercube, and Monte Carlo).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-24-Custom-Distribution-Functions.wmv Custom-Distribution-Functions.wmv]. The model file created during the webinar is [[media:Custom Distribution Functions.ana|Custom Distribution Functions.ana]].

=== [[Regular Expressions]] ===

'''Date and Time:''' Thursday, 9 July 2009, 10:00am - 11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Analytica 4.2 exposes a new and powerful ability to utilize Perl-Compatible regular expressions for text expression analysis. This feature has particular applicability for parsing application when importing data. Long known as the feature that makes Perl and Python popular as data file processing languages, that same power is now readily available within Analytica's [[FindInText]], [[SplitText]], and [[TextReplace]] functions.

This talk will only touch on the regular expression language itself (information on which is readily available elsewhere), but instead focuses on the use of these expressions from the Analytica expressions, especially the extracting of text that matches to subpatterns and finding repeated matches.

One relatively complex example that I plan to work through is the parsing of census population data from datafiles downloaded from the U.S. census web site. The task includes parsing highly variable HTML, as well as multiple CSV files with formatting variations that occur from element to element. These variations, which are typical in many sources of data, demonstrate why the flexibility of regular expressions can be extremely helpful when parsing data files.

Regular expressions themselves are extremely powerful, but when overused, can be very cryptic. So even though it is possible to get carried away with this power, it is good to know how to balance the temptation.

This talk is appropriate for moderate to advanced level modelers.

A recording of this webinar can be watched at [http://AnalyticaOnline.com/WebinarArchive/2009-07-09-Regular-Expressions.wmv Regular-Expressions.wmv]. If you are new to regular expressions, I've included a slides on the regular expression patterns that I made use of in [[media:Regular_Expressions.pps|this power point show]] (these were not shown during the webinar). The model file developed during the webinar is [[media:Regular expressions.ana|Regular expressions.ana]].

=== Using the [[Check Attribute]] to validate inputs and results ===

''' Date and Time:''' Thursday, 17 July 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Check Attribute|check attribute]] provides a way to validate inputs and computed results. When users of your model are entering data, this can provide immediate feedback when they enter values that are out of range or inconsistent. When applied to computed results, it can help catch inconsistencies, which can help reduce error rates and accidental introduction of errors later.

In this talk, I'll demonstrate how to define a check validation for a variable, and how to turn on the check attribute visible so that it is visible in the object window. I'll demonstrate how the failed check alert messages can be customized. And perhaps most interestingly, how the check can be used in edit tables for cell-by-cell validation, so that out-of-range inputs are flagged with a red background, and alert balloons pop-up when out-of-range inputs are entered. Cell-by-cell validation when certain restrictions on the check expression are followed, which I'll discuss.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-17-Check-Attribute.wmv Check-Attribute.wmv] (Note: There is audio, but screen is black, for first 50 seconds). The model used during this webinar, with the check attributes inserted, is at [[media:Check attribute -- car costs.ana|Check attribute -- car costs.ana]].

=== The Performance Profiler ===

'''Date and Time:''' October 9, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''Requires Analytica Enterprise''

When you have a model that takes a long time to compute, thrashes in virtual memory, or uses up available memory, the Performance Profiler can tell you where your model is spending its time and how much memory is being consumed by each variable to cache results. It is not uncommon to find that even in a very large model, a small number (e.g., 2 to 5) of variables account for the lion's share of time and memory. With this knowledge, you can focus your attention optimizing the definition of those few variables. On several occassions I've achieved more than 100-fold sped up in computation time on large models using this technique.

The Performance Profiler requires with Analytica Enterprise or Optimizer. I'll demonstrate how to use the profiler with some basic discussions of what is does and does not measure. One neat aspect of the profiler is that you can actually activate it after the fact. In otherwords, even though you haven't adding profiling to your model, if you happen to notice something taking a long time, you can add it in to find out where the time was spent.

Using the Profiler is pretty simple, so I expect this session will be somewhat shorter than usual. The content will be oriented primarily to people who are unfamiliar with the profiler, although I will also try to provide some behind the scenes details and can answer questions about it for

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-10-09-Performance-Profiler.wmv Performance-Profiler.wmv]. The model file containing the first few examples from the webinar can be downloaded from [[media:Simple Performance Profiler Example.ana|Simple Performance Profiler Example.ana]].

= Organizing Models =

=== Modules and Libraries ===

'''Date and Time:''' 10 Dec 2009 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Modules form the basic organizational principle of an Analytica model, allowing models to be structured hierarchically, keeping things simple at every level even in very large complex models. You can use linked modules to store your model across multiple files. This capability enables reuse of libraries and model logic across different models, and allows you to divide your model into separate pieces so that different people can work concurrently on different pieces of the model.

In this talk, I will review many aspects of modules and libraries. We'll see how to use linked modules effectively. I'll cover what the the distinctions are between Modules, Libraries, Models and Forms. I'll demonstrate various considerations when adding modules to existing models -- such as whether you want to import system variables or merge (update) existing objects, and some variations on what is possible there. We'll see how to change modules (or libraries) from being embedded to linked, or vise versa, and how to change the file location for a linked module. When distributing a model consisting of multiple module files, I'll go over directory structure considerations (the relative placement of module files), and also demonstrate how you can store a copy of your model with everything embedded in a single file for easy distribution.

I'll also discuss definition hiding and browse-only locking. By locking individual modules, you can create libraries with hidden and unchangeable logic that can be used in the context of other people's models, keeping your algorithms hidden. Or, you can distribute individual models that are locked as browse only, even in the context of a larger model where the remainder of the model is editable.

I'll talk about using linked modules in the context of a source control system, which is often of interest for projects where multiple people are modifying the same model. I'll also reveal an esoteric feature, the Sys_PreLoadScript attribute, and how this can be used to implement your own licensing and protection of intellectual property.

This webinar is appropriate for all levels of Analytica model builders.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-12-10-Linked-Modules.wmv Linked-Modules.wmv]. The starting model used in the webinar can be downloaded from [[media:Loan_policy_selection_start.ana|Loan_policy_selection_start.ana]], and then you can follow along to introduce and adjust modules as depicted in the recording if you like.

= Uncertainty & Probability Topics =

=== Gentle Introduction to Modeling Uncertainty: Webinar Series ===

'''Date and Time:'''
:'''Session 1:''' Thursday, 29 Apr 2010 10:00am Pacific Daylight Time
:'''Session 2:''' Thursday, 6 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Are you someone who has never built a model containing explicit representions of uncertainty? Did that Statistics 1A class you took a long time ago instill a belief that probability distributions are irrelevant to the type stuff you work on? Are you afraid to start representing uncertainty explicitly because you just don't have the statistics background and don't know much about probability and probability distributions?

If any of these sentiments resonate with you, then this webinar (series) may be for you.

These are interactive webinars. Be prepared to answer some questions, and have Analytica fired up in the background. You are going to use it to compute the answer to a couple exercises! Even if you are watching the recording, be ready to complete the exercises.

This webinar series is most appropriate for:
* Beginning Analytica model builders.
* Users of models that present results with uncertainty.
* Accomplished spreadsheet or Analytica model builders who have not previously incorporated uncertainty.
* People looking to learn the basics of probability for the representation of uncertainty.

==== Session 1: Uncertainty and Probability ====

In the first session discusses different sources and types of uncertainty, probability distributions and how they can be used to represent uncertainty, various different interpretations of probabilities and probability distributions, and reasons why it is valuable to represent uncertainty explicitly in your quantitative models.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2010-04-29-Modeling-Uncertainty1.wmv Modeling-Uncertainty1.wmv]. A copy of the model created by the presenter during the webinar (the scholarship example) can be downloaded from [[media:Modeling uncertainty 1 - princeton scholarship.ana|Modeling uncertainty 1 - princeton scholarship.ana]]. Power point slides can be downloaded from: [[media:Modeling Uncertainty 1.ppt|Modeling Uncertainty 1.ppt]].

==== Session 2: Probability Distributions ====

How do you characterize the amount of uncertainty you have regarding a real-valued quantity? This second session explores this question, and introduces the concepts of average deviation (aka absolute deviation), variance and standard deviation. It then introduces the concept of a ''probability distribution'' and the [[Normal]] and [[LogNormal]] distributions. We examine the [http://en.wikipedia.org/wiki/EVIU expected value of including uncertainty] and do a few modeling exercises that demonstrate how it can be highly misleading, even expensive, to ignore uncertainty.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-06-Prob-Distributions.wmv Prob-Distributions.wmv]. The model build during the webinar can be downloaded from [[media:Probability Distributions Webinar.ana|Probability Distributions Webinar.ana]]. Power point slides are at [[media:Modeling Uncertainty 2.ppt|Modeling Uncertainty 2.ppt]].

==== Session 3: Monte Carlo ====

'''Date and Time:''' Thursday, 13 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

In this third webinar in the "Gentle Introduction to Modeling Uncertainty" series, we will see how a probability distribution can be represented as a set of representative samples, and how this leads to a very general method for propagating uncertainty to computed results. This method is known as Monte Carlo simulation.

Analytica represents uncertainty by storing a representative sample, so we'll be learning about how Analytica actually carries out uncertainty analysis. We explore how all the uncertainty result views in Analytica are created from the sample, and learn various 'tricks' for nice histograms for PDF views in various situations.

We'll learn about the [[Run]] index, and how this places samples across different variables in correspondence. We'll learn about the generality of the Monte Carlo for propagating uncertainty, and also learn what Latin Hypercube sampling is.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-13-Monte-Carlo.wmv Monte-Carlo.wmv]. The power point slides are at: [[media:Monte Carlo Simulation.ppt|Monte Carlo Simulation.ppt]]. Example models created during the webinar include: [[media:Representing Uncertainty - Mining Example.ana|Mining Example.ana]], [[media:Representing Uncertainty - Explicit samples.ana|Explicit samples.ana]]and [[media:Representing Uncertainty 3 - Misc.ana|Representing Uncertainty 3 - Misc.ana]] (product of normals and comparison between Latin Hypercube and Monte Carlo precision).

==== Session 4: Measures of Risk and Utility ====

'''Date and Time:''' Thursday, 20 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This fourth webinar in the "Gentle Introduction to Modeling Uncertainty" series will explore concepts and quantitative measures of Risk and Utility. We'll discuss various conceptions and types of risk, and explore topics relevant to model-building that include utility and loss functions, expected value, expected utility, risk neutrality, risk aversion, fractiles and Value-at-risk (VaR).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-20-Risk-And-Utility.wmv Risk-And-Utility.wmv]. The power point slides can be viewed at [[media:Measures of Risk and Utility.ppt|Measures of Risk and Utility.ppt]]. There is an interesting modeling exercise and exploration of Expected Shortfall near the end of the power point slides that was not covered during the webinar. The worked out model examples from the webinar, along with a solution to the final example not covered, can be downloaded from [[media:Measures of Risk.ana|Measures of Risk.ana]].

==== Session 5: Risk Analysis for Portfolios ====

'''Date and Time:''' Thursday, 3 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Committing to a single project or investing in a single asset entails a certain amount of risk along with the potential payoff. If you are able to proceed with multiple projects or invest in multiple assets, the degree of risk may be reduced substantially with small impact on potential return. In this fifth webinar in the "Gentle Introduction to Modeling Uncertainty" series, we'll look at modeling portfolios, such as portfolios of investments or portfolios of research and development projects, and the impact this has on risk and return. Portfolio analysis is the basis for practices such as diversification and hedging, and is a key of risk management.

As with other topics in this webinar series, the presentation and discussion is designed for people who are new to the use of these concepts in a model building context.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2010-06-03-Portfolio-Risk.wmv Portfolio-Risk.wmv]. The Power Point slides are at [[media:Risk Analysis for Portfolios.ppt|Risk Analysis for Portfolios.ppt]]. These include some exercises at the end (for homework!) not covered during the webinar, including continuous portfolio allocations. The model developed during the webinar, augmented to include answers to additional exercises is at [[media:Risk Analysis for Portfolios.ana|Risk Analysis for Portfolios.ana]].

==== Session 6: Common Parametric Distributions ====

'''Date and Time:''' Thursday, 10 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

During the first five sessions of the ''Gentle Introduction to Modeling Uncertainty'' webinar series, you have been introduced to three distribution functions: [[Bernoulli]], [[Normal]] and [[LogNormal]]. In this webinar, we're going to increase this repertoire and learn about other common parametric distributions. I'll discuss situations where specific distributions are particularly convenient or natural for expressing uncertainty about certain types of quantities, and other reasons for why you might prefer one particular distribution type over another. We'll also examine the distinction between discrete and continuous distributions.

As with other topics in this webinar series, the presentation and discussion is designed for people who are new to the use of these concepts in a model building context.

A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-06-10-Parametric-Distributions.wmv Parametric-Distributions.wmv]. The power point slides are at [[media:Common Parametric Distributions.ppt|Common Parametric Distributions.ppt]], and the Analytica model containing the exercises and solutions to exercises not covered during the live recording is at [[media:Common-Parametric-Distributions.ana|Common-Parametric-Distributions.ana]].

==== Session 7: Expert Assessment of Uncertainty ====

'''Date and Time:''' Thursday, 24 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

For most uncertainty analysis, uncertainties about many key quantities must be assessed by expert judgment. There has been a lot of empirical research on human abilities to express their knowledge and uncertainty in the form of probability distributions. It shows that we are liable to a variety of biases, such as overconfidence and motivational biases. I'll give an introduction to practical methods developed by decision analysts to avoid or minimize these biases. I'll give some examples from recent work in expert elicitation for the Department of Energy on the future performance of renewable energy technologies.
I'll also discuss ways to aggregate judgments from different experts.

The session is appropriate for people who are new to this area. This probably includes just about everybody!

This session will draw from Chapters 6 and 7 of "Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis" by M Granger Morgan & Max Henrion, Cambridge University Press, 1992

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2010-06-24-Expert-Assessment.wmv Expert-Assessment.wmv]. (At this time, there seems to be a problem with the uploaded file. We'll attempt to re-upload it, hopefully on Fri 16 July 2010).

==== Session 8: Hypothesis Testing ====

'''Date and Time:''' Thursday, 15 July 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Hypothesis testing from classical statistics addresses the question of whether the apparent support for a given hypothesis is statistically significant. In the field of classical statistics, this is perhaps the most heavily emphasized application of probability concepts, and the methodology is used (if not required by editors) when publishing results for research studies in nearly every field of empirical study.

To illustrate the basic idea, suppose a journalist selects 10 Americans at random and asks whether they support a moratorium on deep sea drilling. Seven of the 10 respond "yes", so the the next day he publishes his article "The Majority of Americans Support a Moratorium on Deep Sea Drilling". His sample is certainly consistent with this hypothesis, but his conclusion is not credible because with such a small sample, this majority could have easily been a random quirk (sampling error). Hence we would say that the conclusion is not "''statistically significant''". But how big does the sample have to be to achieve statistical significance? Where should we draw the line when determining whether the data's support is statistically significant? These are the types of questions addressed by this area of statistics.

Hypothesis testing is a central topic in every introductory Statistics 1A course, often comprising more than half of the total course syllabus. But most introductory courses emphasize a cookbook approach in favor of a conceptual understanding, apparently in the hope of providing people in non-statistical fields step-by-step recipes to follow when they need to publish results in their own fields. As a result, the methodology is possibly misused more often than it is applied correctly, and published results are commonly misinterpreted.

In this seminar, I intend to emphasize a conceptual understanding of the statistical hypothesis methodology rather than the more traditional textbook methodology. After this webinar, when you read "our hypothesis was confirmed by the data at a p-value=0.02 level", or "the hypothesis was rejected with a p-value of 0.18", you should be able to precisely relay what these statements really do or do not imply. You should understand what a p-value and confidence level really denote -- they do not represent, as many people think, the probability that the hypothesis is true.

We will also, of course, examine how we can carry out computations of significance levels (i.e., p-values) within Analytica. Statistics texts are filled with numerous "standard" hypothesis tests (e.g., t-tests, etc), each based on a specific set of assumptions. In this webinar, we'll dive into this in a more general way, where we get to start with our own set of arbitrary assumptions, leveraging the power of Monte Carlo for computation. This means there are no recipes to remember, you can compute significance levels for any statistical model, even if the same assumptions don't appear in your statistics texts, and most importantly, you'll be left with a more general understanding of the concepts.

As a prerequisite, this webinar will assume little more than the introductory background from the earlier webinars in this "Gentle Introduction to Modeling Uncertainty" series. It is appropriate for people who have never taken a Statistics 1A course, or for the majority of people who have taken that introduction to Statistics but could use a refresher.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2010-07-15-Hypothesis-Testing.wmv Hypothesis-Testing.wmv]. To follow along with the webinar, you'll want to also download the Analytica model file [[media:Hypothesis Test S&P Volatility.ana|Hypothesis Test S&P Volatility.ana]] before staring. You'll use the data in that model for the various exercises during the webinar.

Solutions to exercises are saved in this version of the model (created during the webinar): [[media:Hypothesis Test S&P Volatility solution.ana|Hypothesis Test S&P Volatility solution.ana]]. I also inserted a solution to the Parkinson's data test that wasn't covered in the webinar but is contained in the [[media:Hypothesis Testing.ppt|Power Point slides]].

=== Correlated and Multivariate Distributions ===

'''Date and Time:''' Thursday, March 13, 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This talk will discuss various techniques within Analytica for defining probability distributions with specified marginal distributions, and also being correlated with other uncertain variables. Techniques include the use of conditional and hierarchical distributions, multivariate distributions, and Iman-Conover rank-correlated distributions.

The model created during session talk is [[media:Correlated distributions.ana|Correlated distributions.ana]]. You can watch a recording of the webinar from [http://AnalyticaOnline.com/WebinarArchive/2008-03-13-Correlated-Distributions.wmv Correlated-Distributions.wmv].

=== Assessment of Probability Distributions ===

'''Date and Time:''' March 6, 2008 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

''' Abstract'''

When building a quantitative model, we usually need to come up with estimates for many of the parameters and input variables that we use in the model. Because these are estimates, it is good idea to encode these as probability distributions, so that our degree of ''subjective uncertainty'' is explicit in the model. The process of encoding a distribution to reflect the level of knowledge that you (or the experts you work with) have about the true value of the quantity is referred to as ''probability (or uncertainty) assessment'' or ''probability elicitation''.

This webinar will be a highly interactive one, where all attendees are expected to participate in a series of uncertainty assessments as we explore the effects of cognitive biases (such as over-confidence and anchoring), understand what it means to be ''well-calibrated'', and utilize scoring metrics to measure your own degree of calibration. These exercises can help you improve the quality of your distribution assessments, and serve as tools that can help you to when eliciting estimates of uncertainty from other domain experts.

The Analytica model [[media:Probability assessment.ana|Probability assessment.ana]] contains a game of sorts that takes you through several probability assessments and scores your responses. Participants of the webinar played this game by running this model, if you are going to watch the webinar, you will want to do the same. You may want to wait until the appropriate point in the webinar (after preliminary stuff has been covered) before starting. You can watch the webinar recording here: [http://AnalyticaOnline.com/WebinarArchive/2008-03-06-Probability-Assessment.wmv Probability-Assessment.wmv].
The power point slides from the talk are here: [[media:Assessment_of_distributions.ppt|Assessment_of_distributions.ppt]].

=== Statistical Functions ===

'''Date and Time:''' Thursday, 21 Aug 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''This topic was presented in Aug 2007, but not recorded at that time.''

A statistical function is a function that process a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica. I'll describe several built-in statistical functions such as [[Mean]], [[SDeviation]], [[GetFract]], [[Pdf]], [[Cdf]], and [[Covariance]]. I'll demonstrate how all built-in statistical functions can be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index). I'll discuss how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all built-in statistical functions can compute weighted statistics, where each point is assigned a different weight. I'll briefly touch on this feature as a segue into next week's topic, Importance Sampling.

This talk can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-08-21-Statistical-Functions.wmv Statistical-Functions.wmv]. The model built during this talk is available for download at [[media:Intro to Statistical Functions.ana|Intro to Statistical Functions.ana]].

=== [[RankCorrel|Spearman Rank Correlation]] ===

'''Date and Time:''' Thursday, 25 March 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Many measures for quantifying the degree of statistical dependence between quantities are used in statistics. THe two most commonly used are [[Correlation|Pearson's Linear Correlation]] and [[RankCorrel|Spearman's Rank Correlation]], computed respectively in Analytica by the functions [[Correlation]] and [[RankCorrel]]. Pearson's [[Correlation]], which is what people usually mean when they just use the term "Correlation", is a measure of how linear the relationship between two variables is. Spearman's [[RankCorrel|Rank Correlation]] is a measure of monotonic the relationship between two variables is.

This talk provides an introduction to the concept of [[RankCorrel|rank correlation]], how it is distinguished from standard [[Correlation|Pearson correlation]], and what it measures. There are several notable and rather diverse uses of [[RankCorrel]], which include these (and probably many others):
* A quantitative measure of the degree to which two variables are monotonically related. (E.g., the degree to which an increase in one leads to an increase, or decrease, in the other).
* Testing (from measurements) whether two factors are statistically dependent
* Importance analysis: Determining how much the uncertain of an input contributes to the uncertainty of an output.
* Sampling from joint distributions with arbitrary marginals and specified rank-correlations ([[Correlate_With]] and [[Correlate_Dists]])

I will focus mostly on the first two factors in this talk (previous webinars on [[#Sensitivity_Analysis_Topics|Sensitivity Analysis]] have covered the Importance Analysis usage to some extent, and a previous webinar on [[#Correlated_and_Multivariate_Distributions|Correlated and Multivariate Distributions]] has covered the last point).

Standard hypothesis tests exist for determining whether two factors are statistically dependent by testing the hypothesis that their [[RankCorrel|rank correlation]] is non-zero (null hypothesis that it is zero). When the P-value of these tests is less than 5% (or 1%), you would be justified in concluding that the two variables are statistically dependent. I will demonstrate how to compute this P-value.

Then I will introduce a new analysis of [[RankCorrel|rank correlation]] that I came up with, which I think is novel and potentially pretty useful, somewhat related to the classical hypothesis tests just mentioned. Suppose you gather a small sample of data on two variables in a study and you want to determine how strong the monotonicity between the two variables is. You can compute the ''sample rank correlation'' for the data set, but this is only an estimate since you have a small sample size and thus sampling error may throw off this estimate. So suppose we imagine there is some "true" underlying rank correlation between the variables (this in itself is a new concept, which I will make precise). From your data set, you have some knowledge about the true value of this underlying rank correlation -- the larger your sample size, the more precise your knowledge is. The new technique I describe here computes a (posterior) distribution over the true underlying rank correlation, from which you can express your rank correlation result as a range (such as rc=0.6±0.2), and answer questions such as what is the probability that the underlying rank correlation is between -0.1 and 0.1, P(-0.1 < rc < 0.1), or P(rc>0), etc. Although this is essentially a posterior distribution, there is no prior distribution involved or needed to computate it, so it is simply a function of the measured data and of the sample size. It really is a probability distribution on the underlying rank correlation, not just a P-value, making it much more useful.

This new analysis is also useful for quantifying the probability that two factors are independent in a manner not possible with the classical tests. The classical P-value of the aforementioned tests measure the probability of a Type II error for the hypothesis that variables are dependent. These tests do not provide the probability of a Type I error, which would be the criteria for concluding that a claim of statistical independence is statistically signficant. This new measure, however, can justifiably be used for quantifying a claim of statistical independence since it allows P(-c<rc<c) to be computed for any c.

I will demonstrate how this new analysis of rank correlation works and is encoded within Analytica, and show how to read off the interesting results.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2010-03-25-Rank-Correlation.wmv Rank-Correlation.wmv]. The model files created during the talk are available at: [[media:Rank-Correlation-Examples.ana|Rank-Correlation-Examples.ana]] and [[media:Rank-Correlation-Analysis.ana|Rank-Correlation-Analysis.ana]].

=== Statistical Functions in Analytica 4.0 ===

Date and Time: Thursday, Aug 16, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

A statistical function is a function that process a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica 4.0. In Analytica 4.0, all built-in statistical functions can now be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index), eliminating the need for separate function libraries. I will demonstrate this use, as well as several new statistical functions, e.g., [[Pdf]], [[Cdf]], [[Covariance]]. I will explain how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all statistical functions in Analytica 4.0 can compute weighted statistics, where each point is assigned a different weight. I'll cover the basics of sample weighting, and demonstrate some simple examples of using this for computing a Bayesian posterior and for importance sampling from an extreme distribution.

The Analytica model file that had resulted by the end of the presentation can be downloaded here:
[[media:User Group Webinar - Statistical Functions.ANA | User Group Webinar - Statistical Functions.ANA]].

=== The [[Large Sample Library: User Guide|Large Sample Library]] ===

'''Date and Time:''' Thursday, 18 Feb 2010 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Large Sample Library: User Guide|Large Sample Library]] is an Analytica library that lets you run a Monte Carlo simulation for a large model or a large sample size that might otherwise exhaust computer memory, including virtual memory. It breaks up a large sample into a series of batch samples, each small enough to run in memory. For selected variables, known as the ''Large Sample Variables'' or ''LSVs'', it accumulates the batches into a large sample. You can then view the probability distributions for each LSV using the standard methods — confidence bands, PDF, CDF, etc. — with the full precision of the large sample.

Memory is saved by not storing results for non-LSVs.

This presentation introduces this library and how to use it.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2010-02-18-Large-Sample-Library.wmv Large-Sample-Library.wmv]. The Large Sample library can be downloaded for use in your own models from the [[Large Sample Library: User Guide]] page. The two example models used during this webinar were: [[media:Enterprise model3.ANA|Enterprise model3.ana]] and [[media:Simple example for Large Sample Library.ana|Simple example for Large Sample Library.ana]].

= Sensitivity Analysis Topics =

=== Tornado Charts ===

'''Time and Date:''' Thursday, 20 Mar 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract:'''

[[Image:Tornado plot.png]]

A tornado chart depicts the result of a '''local sensitivity analysis''', showing how much a computed result would change if each input were varied one input at a time, with all other inputs held to their baseline value. The result is usually plotted with horizontal bars, sorted with larger bars on top, resulting in a graph resembling the shape of a tornado, hence the name. There a numerous variations on tornado charts, resulting from different ways of varying the inputs, and in some cases, different metrics graphed.

This talk will walk through the steps of setting up a Tornado chart, and explore different variations of varying inputs. We'll also explore some more complex issues that can arise when some inputs are arrays.

The model used during this talk is here: [[media:Tornado Charts.ANA|Tornado Charts.ana]] (the stuff for the talk was in the Tornado Analysis module). You can watch a recording of this webinar from [http://AnalyticaOnline.com/WebinarArchive/2008-03-20-Tornado-Charts.wmv Tornado-Charts.wmv].

=== Advanced Tornado Charts -- when inputs are Array-Valued ===

'''Date and Time:''' Thursday, April 17, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The webinar of 20-Mar-2008 ([http://AnalyticaOnline.com/WebinarArchive/2008-03-20-Tornado-Charts.wmv Tornado-Charts.wmv], see webinar archives) went through the fundamentals of setting up a local sensitivity analysis and plotting the results in the form of a tornado chart. That webinar also discussed the many variations of tornado analyses (or more generally, local sensitivity analyses) that are possible.

This talk builds on those foundations by going a step further and addressing tornado analyses when some of the input variables are array-valued. The presence of array-valued inputs introduces many additional possible variations of analyses, as well as many modeling complications. For example, a local sensitivity analysis varies one input at a time, but that could mean you vary each input variable (as a whole) at a time, or it could mean that you vary each cell of each input array individually. Either is possible, each resulting in a different analysis. Some of these variations compute the correct result automatically through the magic of array abstraction, once you've set up the basic tornado analysis that we covered in the first talk, while other require quite a bit of additional modeling effort. However, even the ones that produce the correct result can often be made more efficient, particularly when the indexes of each input variable are different across input variables.

When we do opt to vary input arrays one cell at a time, the display of the results may be dramatically effected. Although we can keep the results in an array form, the customary tornado chart require us to ''flatten'' the multi-D arrays and label each bar on the chart with a cell coordinate.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-04-17-Tornados-With-Arrays.wmv Tornados-With-Arrays.wmv]. This webinar made use of the following models: [[media:Sales Effectiveness Model with tornado.ana|Sales Effectiveness Model with tornado.ana]], [[media:Biotech R&D Portfolio Tornado.ana|Biotech R&D Portfolio with Tornado.ana]], [[media:Sensitivity Analysis Library.ana|Sensitivity Analysis Library.ana]], and [[media:Sensitivity Functions Examples.ana|Sensitivity Functions Examples.ana]]. See [[The Sensitivity Analysis Library]] for more information on how to use [[media:Sensitivity Analysis Library.ana|Sensitivity Analysis Library.ana]] in your own models.

= Financial Analysis =

=== Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]]) ===

'''Date and Time:''' 18 Dec 2008, 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This is Part 3 of a multi-part webinar series where we have been covering the modeling and evaluation of cash flows over time in an interactive exercise-based webinar format, where concepts are introduced in the form of modeling exercises, and participants are asked to complete the exercises in Analytica during the webinar. Part 3 covers Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]]), and includes seven modeling exercises.

To speed the presentation up, I am providing the exercises in advance: [[media:NPV_and_IRR.ppt|NPV_and_IRR.ppt]]. I urge you to take a shot at completing them before the webinar begins, and we'll advance through the exercises more rapidly so as to complete the topic material within the hour. By attempting the exercises in advance, you'll have a good opportunity to compare your solutions to mine, and to ask questions about things you got stuck on.

A dollar received today is not worth the same as a dollar received next year. Taking this time-value of money (or more generally, time-value of utility) into account is very important when comparing cash flows over time that result from long-term capital budgeting decisions. Net Present Value (NPV) and Internal Rate of Return (IRR) are the two most commonly used metrics examining the effective value of an investment's cash flow over time. Both concepts are pervasive in decision-analytic models.

This webinar will be highly interactive. Fire up a instance of Analtyica as you join on. As I introduce each concept, I'll provide you with cash flow scenarios, and give you a chance to compute the result yourself using Analytica. This talk is intended for people who are not already well-versed in NPV and IRR, or for people who already have a good background with those concepts but are new to Analytica and thus can learn from the interactive practice of addressing these exercises during the talk.

See also the materials from Parts 1 and 2 (Net Present Value, 20 Nov 2008 and 4 Dec 2008) elsewhere on this page. This session begins with the model [[media:Cash Flow Metrics 2.ana|Cash Flow Metrics 2.ana]], and ends with [[media:Cash Flow Metrics 3.ana|Cash Flow Metrics 3.ana]]. You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-12-18-IRR.wmv IRR.wmv].

=== Bond Portfolio Analysis ===

'''Date and Time: ''' 11 Dec 2008, 10:00am Pacific Standard Time

'''Presenter:''' Rob Brown, Incite! Decision Technologies

'''Abstract'''

I demonstrate how to value a bond portfolio in which bonds are bought and sold on an uncertain frequency. The demonstration shows how Intelligent Arrays and related functions can greatly simplify calculations of multiple dimensions that would typically require multiple interconnected sheets in a spreadsheet or nested do-loops in a procedural language.

You can watch a recording of this webinar at
[http://AnalyticaOnline.com/WebinarArchive/2008-12-11-Bond-Portfolio-Analysis.wmv Bond-Portfolio-Analysis.wmv]. The model underlying the presentation is [[media:Bond Portfolio Valuation.ana|Bond Portfolio Valuation.ana]], and the power point slides are at [[media:Bond Portfolio Valuation.ppt|Bond Portfolio Valuation.ppt]].

=== Net Present Value ([[NPV]]) ===

'''Date and Time:''' Part I : Thursday, 20 Nov 2008, 10:00am Pacific Standard Time
::::Part II : Thursday, 4 Dec 2008, 10:00am Pacific Standard Time

(Parts 1 & 2 cover NPV -- part 3, listed now separately, covers IRR)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A dollar received today is not worth the same as a dollar received next year. Taking this time-value of money (or more generally, time-value of utility) into account is very important when comparing cash flows over time that result from long-term capital budgeting decisions. Net Present Value ([[NPV]]) and Internal Rate of Return ([[IRR]]) are the two most commonly used metrics examining the effective value of an investment's cash flow over time. Both concepts are pervasive in decision-analytic models.

This multi-part webinar provides an introduction to the concepts of present value, discount rate, [[NPV]] and [[IRR]]. We'll discuss the interpretation of ''discount rate'', and we'll get practice computing these metrics in Analytica. We'll examine the pitfalls of each metric, and we'll examine the interplay of each metric with explicitly modelled uncertainty (including the concepts of Expected NPV (ENPV) and Expected IRR (EIRR)).

This webinar will be highly interactive. Fire up a instance of Analtyica as you join on. As I introduce each concept, I'll provide you with cash flow scenarios, and give you a chance to compute the result yourself using Analytica. This talk is intended for people who are not already well-versed in [[NPV]] and [[IRR]], or for people who already have a good background with those concepts but are new to Analytica and thus can learn from the interactive practice of addressing these exercises during the talk.

I have assembled quite a bit of material, which I believe will fill two webinar sessions. Part 1 will focus mostly on present value, NPV, discount rate, and the use of NPV with uncertainty. Part 2 will focus mostly on IRR, several "gotchas" with IRR, and MIRR.

Materials:
* [[media:Cash Flow Metrics 2.ana|Cash Flow Metrics 2.ana]] : Model at end of second session
* [[media:Cash Flow Metrics 1.ANA|Cash Flow Metrics 1.ana]] : Model at end of first session
* [http://AnalyticaOnline.com/WebinarArchive/2008-11-20-NPV-and-IRR1.wmv NPV-and-IRR1.wmv] : Webinar recording of Part 1.
* [http://AnalyticaOnline.com/WebinarArchive/2008-12-04-NPV-and-IRR2.wmv NPV-and-IRR2.wmv] : Webinar recording of Part 2.

Note: Part 1 covered 5 exercises, covering present value, discount rate, modeling certain cash flows, computing NPV, and graphing the NPV curve. Part 2 added exercises 6-9, covering cash flows at non-uniformly-spaced time periods, valuating bonds and treasury notes, cash flows with uncertainty, and using the CAPM to find invester-implied corporate discount rate.

The "class" will continue with Part 3 beginning with Internal Rate of Return.

= Data Analysis Techiques =

=== Statistical Functions ===

'''Date and Time:''' Thursday, May 22, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A statistical function is a function that processes a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica 4.0. In Analytica 4.0, all built-in statistical functions can now be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index), eliminating the need for separate function libraries. I will demonstrate this use, as well as several new statistical functions, e.g., [[Pdf]], [[Cdf]], [[Covariance]]. I will explain how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all statistical functions in Analytica 4.0 can compute weighted statistics, where each point is assigned a different weight. I'll cover the basics of sample weighting, and demonstrate some simple examples of using this for computing a Bayesian posterior and for importance sampling from an extreme distribution.

This talk is appropriate for moderate to advanced users.

A recording of this webinar can be watched at
[http://AnalyticaOnline.com/WebinarArchive/2008-05-22-Statistical-Functions.wmv Statistical-Functions.wmv].
The model created during this webinar is at [[media:Statistical Functions.ANA|Statistical Functions.ana]].

=== Principal Components Analysis (PCA) ===

'''Date and Time:''' 15 Jan 2009, 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Principal component analysis (PCA) is a widely used data analysis technique for dimensionality reduction and identification of underlying common factors. This webinar will provide a gentle introduction to PCA and demonstrate how to compute principal components within Analytica. Intended to be at an introductory level, with no prior experience with PCA (or even knowledge of what it is) assumed.

The model developed during this talk, where the principal components were computed for 17 publically traded stocks based on the previous 2 years of price change data is [[media:Principal Component Analysis.ana|Principal Component Analysis.ana]]. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-01-15-PCA.wmv PCA.wmv].

=== Variable Stiffness Cubic Splines ===

'''Date and Time:''' 2 October 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Brian Parsonnet, ICE Energy

'''Abstract'''

The Variable Stiffness Cubic Spline is a highly robust data smoothing and interpolation technique. A stiffness parameter adjusts the variability of the curve. At the extreme of minimal stiffness, the curve approaches a cubic spline (like [[CubicInterp]]) that passes through all data points, while at the other extreme of maximal stiffness, the spline curve becomes the best-fit line. Weight parameters can be used to constrain the curve to include selected points, while smoothing over others. The first, second and third derivatives all exist and are readily available.

I'll introduce and demonstrate [[User-Defined Functions]] that compute the variable stiffness cubic spline and interpolate to new points. I'll also show how these curves can be used to detect or eliminate anomalies in data.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-10-02-Variable-Stiffness-Cubic-Splines.wmv Variable-Stiffness-Cubic-Splines.wmv]. The model and library with the vscs functions will be posted here within a few weeks.

=== Using [[Regression]] ===

'''Date and Time:''' Thursday, May 1, 2008 at 10:00 - 11:00 Pacific Daylight Time

'''Date and Time:''' Thursday, Aug 30, 2007 at 10:00 - 11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Regression analysis is a statistical technique for curve fitting, discovering relationships in data, and testing hypotheses between variables. In this webinar, I will focus on generalized linear regression, which is provided by Analytica's [[Regression]] function, and examine many ways in which is can be used, including fitting simple lines to data, polynomial regression, use of other non-linear terms, and fitting of autoregressive models (e.g., ARMA). I'll examine how we can assess how likely it is the data might have been generated from the particular form of the regression model used. We can also determine the level of uncertainty in our inferred parameter values, and incorporate these uncertainties into a model that uses the result of the regression. The talk will cover Analytica 4.0 functions [[Regression]], [[RegressionDist]], [[RegressionFitProb]], and [[RegressionNoise]].

You can watch a recording of the 1 May 2008 webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-05-01-Regression.wmv Regression.wmv]. The model developed during that webinar is here: [[media:Using_Regression.ANA|Using Regression.ana]]

=== Logistic Regression ===

'''Date and Time:''' Thursday, 5 June 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

''' Abstract'''

''(Features covered in this webinar require Analytica Optimizer)''

Logistic regression is a technique for fitting a model to historical data to predict the probability of an event from a set of independent variables. In this talk, I'll introduce the concept of Logistic regression, explain how it differs from standard [[Regression|linear regression]], and demonstrate how to fit a logistic regression model to data in Analytica. Probit regression is for all practical purposes the same idea as Logistic regression, differing only in the specific functional form for the model. Poisson regression is also similar except is appropriate when predicting a probability distribution over a dependent variable that represents integer "counts". All are examples of generalized linear models, and after reviewing these forms of logistic regression, it should be clear how other generalized linear model forms can be handled within Analytica.

This topic is appropriate for advanced modelers. I will assume familiarity with [[Regression|regression]] (see the earlier talk on the topic), but will not assume a previous knowledge of logistic regression.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2008-06-05-Logistic-Regression.wmv Logistic-Regression.wmv]. The model developed during this webinar can be downloaded from [[media:Logistic_regression_example.ana|Logistic_regression_example.ana]]. You'll also need the file [[media:BreastCancer.data|BreastCancer.data]].

= Bayesian Techniques =

=== Bayesian Posteriors using Importance Sampling ===

'''Date and Time:''' Thursday, September 4, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Several algorithms for computing Bayesian posterior probabilities are special cases of ''importance sampling''. The [[#Importance_Sampling_.28Rare_events.29|webinar of the previous week, Importance Sampling (rare events)]] introduced importance sampling, covered the theory behind it, how it is applied, and how Analytica's sample weighting feature can be use for importance sampling. This webinar continues with importance sampling, this time exploring how it can be used (at least in some cases) to compute Bayesian posterior probabilities.

I'll provide an introduction to what Bayesian posterior probabilities are, describe a couple importance sampling-based approaches to computing them, and implement a few examples in Analytica. Importance sampling techniques for computing posteriors have limited applicability -- in some cases they work well, other not. I'll try to characterize what those conditions are.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-09-04-Posteriors_using_IS.wmv Posteriors_using_IS.wmv]. About two-thirds through the presentation, we noticed a result that seemed to be coming out incorrectly. I explain what the problem was and fix it in [http://AnalyticaOnline.com/WebinarArchive/2008-09-04-Posteriors_using_IS_addendum.wmv Posteriors_using_IS_addendum.wmv]. The models used during this presentation can be downloaded from [[media:Posterior sprinklers.ana|Posterior sprinklers.ana]] and [[media:Likelihood weighting.ana|Likelihood weighting.ana]].

=== Importance Sampling (Rare events) ===

'''Date and Time:''' Thursday, 28 Aug 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Importance sampling is a technique that simulates a target probability distribution of interest by sampling from a different sampling distribution and then re-weighting the sampled points so that computed statistics match those of the target distribution. The technique has has applicability when the target distribution is difficult to sample from directly, but where the probability density function is readily available. The technique produces valid results in the large sample size limit for any selection of sampling distribution (provided it is absolutely continuous with respect to the target distribution), but best results (i.e., fastest convergence with smaller sample size) are obtained when a good sampling distribution is used. The technique is commonly used for rare-event sampling, where you want to ensure greater sampling coverage in the tails of distributions, where few samples would occur with standard Monte Carlo sampling. During the talk, we develop a rare event model. It also has applicability to the computation of Bayesian posteriors, and sampling of complex distribution.

In this talk we cover the theory behind importance sampling and introduce the [[SampleWeighting|sample weighting]] mechanism that is built into Analytica. We develop a rare-event model to demonstrate how the weighting mechanism is used to achieve the importance sampling. Next week we'll continue with an example of computing a Bayesian posterior probability.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-08-28-Importance-Sampling.wmv Importance-Sampling.wmv]. The model developed during this talk can be downloaded from: [[media:Importance sampling rare events.ANA|Importance Sampling rare events.ana]].

= Presenting Models to Others =

=== Guidelines for Model Transparency ===

'''Date and Time:''' 19 Feb 2009, 10:00am Pacific Standard Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

What makes Analytica models easy for others to use and understand? I will review some example models that illustrate ways to improve transparency -- or opacity. Feel free to send me your candidates ahead of time!
We'll review some proposed guidelines. I hope to stimulate a discussion about what you think works well or not, and enlist your help in refining these guidelines.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-02-19-Transparency-Guidelines.wmv Transparency-Guidelines.wmv].

=== Creating Control Panels ===

'''Date and Time:''' Thursday, May 29, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

It is quite easy to put together "control panels" or "forms" for your Analytica models by creating input and output nodes for the inputs and outputs of interest to your model end users. This webinar will cover the basic steps involved in creating and arranging these forms, along with some tricks for making the process efficient. We'll cover the different types of input and output controls that are currently available, the use of text nodes to create visual groupings, use of images and icons, and the alignment commands that make the process very rapid. We'll learn how to change colors, and look at the use of buttons very briefly. This talk is appropriate for beginning Analytica users.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-05-29-Control-Panels.wmv Control-Panels.wmv] (required Windows Media Player). The model used during this webinar is at [[media:Building Control Panels.ana|Building Control Panels.ana]].

=== Sneak preview of Analytica Web Publisher ===

'''Date and Time:''' Thursday, February 21, 2008, 10:00 - 11:00 Pacific Standard Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

In this week's webinar, Max Henrion, Lumina's CEO, will provide a sneak preview of the Analytica Web Publisher. AWP offers a way to make Analytica models easily accessible to anyone with a web browser. Users can open a model, view diagrams and objects, change input variables, and view results as tables and graphs. Users will also be able to save changed models, to revisit them in later sessions. Model builders can upload models into AWP directly from their desktop. Usually, AWP directories are password protected, so only authorized users can view and use models. But, we also plan to make a free AWP directory available for people who want to share their models openly.

AWP is nearing release for alpha testing. We will welcome your comments and hearing how you might envisage using AWP.

''This webinar was not recorded.''

= Application Integration Topics =

=== OLE Linking ===

'''Time and Date:''' Thursday, 27 Mar 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract:'''

OLE linking is a commonly used methods for linking data from Excel spreadsheets into Analytica and results from Analytica into Excel spreadsheets. It can be used with other applications that support OLE-linking as well. The basic usage of OLE linking is pretty simple -- it is a lot like copy and paste. This webinar covers basics of using OLE linking of fixed-sized 1-D or 2-D tables. I also demonstrate the basic tricks you must go through to link index values and multi-D inputs and outputs. In addition, we discuss what some of those OLE-link settings actually do, and explain how OLE-connected applications connect to their data sources.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-03-27-OLE-Linking.wmv 2008-03-27-OLE-Linking.wmv].

Note: Another 10 minute fast-paced video (separate from the webinar) demonstrates linking data from Analytica into Excel, computing something from that data, and linking the result back into Analytica: [http://AnalyticaOnline.com/WebinarArchive/OLE-to-Excel-and-back.wmv OLE-to-Excel-and-back.wmv].

=== Querying an OLAP server ===

'''Date and Time:''' Thursday, February 14, 2008, 10:00 - 11:00 Pacific Standard Time 
(''Note: Schedule change from an earlier posting. This is now back to the usual Thursday time. '')

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

In this session, I'll show how the [[MdxQuery]] function can be used to extract multi-dimensional arrays from an On-Line Analytical Processing (OLAP) server. In particular, during this talk we'll query Microsoft Analysis Services using MDX. In this talk, I'll introduce some basics regarding OLAP and Analysis Services, discuss the differences between multi-dimensional arrays in OLAP and Analytica, cover the basics of the MDX query language, show how to form a connection string for [[MdxQuery]], and import data. I'll also show how hierarchical dimensions can be handled once you get your data to Analytica.

''Note: Use of the features demonstrated in this webinar require the Analytica Enterprise or Optimizer edition, or the Analytica Power Player. They are also available in ADE.''

The model created during this webinar is available here: [[media:Using MdxQuery.ana|Using MdxQuery.ana]]. You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-02-14-MdxQuery.wmv MdxQuery.wmv] (requires Microsoft Media Player)

=== Querying an ODBC relational database ===

Date and Time: Thursday, February 7, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this talk I'll review the basics of querying an external relational ODBC database using [[DbQuery]]. This provides a flexible way to bring in data from SQL Server, Access, Oracle, and mySQL databases, and can also be used to read CSV-text databases and even Excel. In this talk, I will cover the topics of how to configure and specify the data source, the rudimentary basics of using SQL, the use of Analytica's [[DbQuery]], [[DbWrite]], [[DbLabels]] and [[DbTable]] functions.

''Note: Use of the features demonstrated in this webinar require the Analytica Enterprise or Optimizer edition, or the Analytica Power Player. They are available in ADE."

You can grab the model created during this webinar from here: [[media:Querying an ODBC relational database.ANA|Querying an ODBC relational database.ana]]. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-02-07-Using-ODBC-Queries.wmv Using-ODBC-Queries.wmv].

=== Calling External Applications ===

Date and Time: Thursday, Oct 18, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

The [[RunConsoleProcess]] function runs an external program, can exchange data with that program, and can be used to perform a computation or acquire data outside of Analytica, that then can be used within the model. I'll demonstrate how this can be used with a handful of programs, and code written in several programming and scripting languages. I'll demonstrate a user-defined function that retrieves historical stock data from a web site.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-10-18-Calling-External-Applications.wmv Calling-External-Applications.wmv] (Requires Windows Media Player)

Files created or used during this webinar can be downloaded:
* [[media:Regular Expression Matching.ANA|Regular Expression Matching.ana]]
* [[media:RegExp.vbs|RegExp.vbs]]
* [[media:Read Historical Stock Data.ana|Read Historical Stock Data.ana]]
* For plotting to gnuplot, these gnuplot command files were used. (Note: You may have to adjust some file paths within these files, and within the model): [[media:Gnuplot-candlesticks.dat|Gnuplot-candlesticks.dat]], [[media:Gnuplot-3dsurface.dat|Gnuplot-3dsurface.dat]]
* [[media:ReadURL.exe|ReadURL.exe]] (for C++/CLR source code, see [[Retrieving Content From the Web]])

The example of retrieving stock data from Yahoo Finance is also detailed in an article here: [[Retrieving Content From the Web]]

=== New Functions for Reading Directly from an Excel File ===

'''Date and Time:''' Thursday, 24 April 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

'''(Feature covered requires Analytica Enterprise or better)'''

Hidden within the new release of Analytica 4.1 are three new functions for reading values directly from Excel spreadsheets: [[OpenExcelFile]], [[WorksheetCell]], [[WorksheetRange]]. These provide an alternative to OLE linking and ODBC for reading data from spreadsheets, which may be more convenient, flexible and reliable in many situations. We have not yet exposed these functions on the Definitions menu or in the Users Guide in release 4.1, since they are still in an experimental stage. I would like know that they have been "beta-tested" in a variety of scenarios before we fully expose them (also, the symmetric functions for writing don't exist yet). In this webinar, I will introduce and demonstrate these functions, after which you can start using them with your own problems.

The model created during this talk is here: [[Media:Functions for Reading Excel Worksheets.ana|Image:Functions for Reading Excel Worksheets.ana]]. It read from the example that comes with Office 2003, to which we added a few range names during the talk, resulting in [[media:SOLVSAMP.XLS|SolvSamp.xls]]. Place the excel file in the same directory as the model. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-04-24-Reading-From-Excel.wmv Reading-From-Excel.wmv].

=== Reading Data from URLs to a Model ===

'''Date and Time:''' Thursday, 27 Aug 2009, 10:00am-11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''Requires Analytica Enterprise''

The new built-in function, [[ReadFromUrl]], can be used to read data (and images) from websites, such as HTTP web pages, FTP pages, or even web services like SOAP. In this webinar, I'll demonstrate the use of this function in several ways, including reading live stock and stock option price data, posting data to a web form, retrieving a text file from an FTP site, supplying user and password credentials for a web site or ftp service, downloading and displaying images including customized map and terrain images, and querying a SOAP web service.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-08-27-ReadFromUrl.wmv ReadFromUrl.wmv]. The model with the examples shown during the webinar is at [[media:Reading_Data_From_the_Web.ana|Reading_Data_From_the_Web.ana]].

=== Using the Analytica Decision Engine (ADE) from ASP.NET ===

'''Date and Time:''' Thursday, April 10, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Fred Brunton, Lumina Decision Systems

The Analytica Decision Engine (ADE) allows you to utilize a model developed in Analytica as a computational back-end engine from a custom application. In this webinar, we'll create a simple active web server application using ASP.NET that sends inputs submitted by a user to ADE, and displays results computed by ADE on a custom web page. In doing this, you will get a flavor how ADE works and how you program with it. If you've never created an active server page, you may enjoy seeing how that is done as well. This introductory session is oriented more towards people who do not have experience using ADE, so that you can learn a bit more about what ADE is and where it is appropriate by way of example.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-04-10-ADE-from-ASPNET.wmv ASP-from-ASPNET.wmv]. To download the program files that were created during this webinar [[Media:WebSite6.rar| Click here]].

= Optimization =

=== [[Analytica_4.3#Structured_Optimization|Introduction to Structured Optimization]] ===

'''Date and Time:''' Thursday, Febrary 24, 2011 at 10:00am PST (1:00pm EST, 6:00pm GMT)

'''Presenter''': Paul Sanford, Lumina Decision Systems

'''Abstract'''

[[Analytica 4.3]] is now available for beta testing and will be released in early March. The new version includes expanded optimization capabilities and simplified workflow for encoding optimization problems. The new '''Structured Optimization''' framework in 4.3 is centered around a new function, [[DefineOptimization]](), which replaces all three of the previous type-specific functions: LPDefine(), QPDefine() and NLPDefine(). It also introduces a new node type, '''Constraint''', which allows you to specify constraints using ordinary expressions. Paul will build up some basic examples using Structured Optimization and field questions from users.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2011-2-24-Structured-Optimization.wmv Structured-Optimization.wmv]. The example models used during this webinar are: [[Media:Beer Distribution LP1.ana|Beer Distribution LP1.ana]],[[Media:Beer Distribution LP2.ana|Beer Distribution LP2.ana]], [[File:Plane Allocation LP.ana]],[[File:polynomial NLP.ana]]

=== Introduction to Linear and Quadratic Programming ===

Date and Time: Thursday, Oct 11, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

This talk is an introduction to linear programming and quadratic programming, and an introduction to solving LPs and QPs from inside an Analytica model (via Analytica Optimizer). LPs and QPs can be efficiently encoded using the Analytica Optimizer functions [[LpDefine]] and [[QpDefine]]. I'll introduce what a linear program is for the sake of those who are not already familiar, and examine some example problems that fit into this formalism. We'll encode a few in Analytica and compute optimal solutions. Although LPs and QPs are special cases of non-linear programs (NLPs), they are much more efficient and reliable to solve, avoid many of the complications present in non-linear optimization, and fully array abstract. Many problems that initially appear to be non-linear can often be reformulated as an LP or QP. We'll also see how to compute secondary solutions such as dual values (slack variables and reduced prices) and coefficient sensitivies. Finally, [[LpFindIIS]] can be useful for debugging an LP to isolate why there are no feasible solutions.

You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-10-11-LP-QP-Optimization.wmv LP-QP-Optimization.wmv] (requires Windows Media Player)

The model file created during this webinar is here: [[media:LP QP User Group.ANA|LP QP Optimization.ana]]

=== Non-Linear Optimization ===

Date and Time: Thursday, Oct 4, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

This talk focuses on the problem of maximizing or minimizing an objective criteria in the presence of contraints. This problem is referred to as a non-linear program, and the capability to solve problems of this form is provided by the Analytica Optimizer via the [[NlpDefine]] function. In this talk, I'll introduce the use of [[NlpDefine]] for those who have not previously used this function, and demonstrate how NLPs are structured within Analytica models. I'll examine various challenges inherent in non-linear optimization, tricks for diagnosing these and some ways to address these. We'll also examine various ways in which to structure models for parametric analyses (e.g., array abstraction over optimization problems), and optimizations in the presence of uncertainty.

You can watch a recording of this session here: [http://AnalyticaOnline.com/WebinarArchive/2007-10-04-Nonlinear-Optimization.wmv Nonlinear-Optimization.wmv]

During the talk, these two models were created:
* [[media:Simple Nonlinear optimization.ANA|Simple Nonlinear Optimization.ana]]
* [[media:Nonlinear asset allocation.ANA|Nonlinear asset allocation.ana]]

= Vertical Applications and Case Studies =

=== Regional Weather Data Analysis ===

'''Date and Time:''' Thursday, 22 April 2010 10:00am Pacific Daylight Time

'''Presenter:''' Brian Parsonnet, Ice Energy

'''Abstract'''

There are numerous sources of weather data on the web. Users of this data face a few common problems: how to gather the data in volume, how to normalize the data regardless of source, and how to analyze the results to generate insight. Analytica is the perfect tool to address all three issues simply and efficiently. A sample model will be shown illustrating some of techniques.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-04-22-Regional-Weather-Analysis.wmv Regional-Weather-Analysis.wmv]. The model shown during the talk is [[media:Weather analysis.ana|Weather analysis.ana]], and the data file used by this model for Burbank weather can be downloaded from [[media:Burbank.zip|Burbank.zip]] (remember to Unzip it first to Burbank.txt). To avoid issues with ownership of the data, the temperatures in this file have been randomized (so the data is not accurate) and other fields zeroed out, but this will still allow you to play with the model and data.

=== Automated Monitoring and Failure Detection ===

'''Date and Time:''' 5 Feb 2009, 10:00am Pacific Standard Time

'''Presenter:''' Brian Parsonnet, ICE Energy

'''Abstract'''

In many complex physical systems, the automatic and proactive detection of system failures can be highly beneficial. Often dozens of sensor readings are collected over time, and a computer analyzes these to detect when system behavior is deviating from normal. Sounding an alert can then facilitate early intervention, perhaps catching a component that is just starting to go bad.

In a complex physical system with multiple operating modes and placed in a changing environment, anomaly detection is a very difficult problem. Simple sensor thresholds (and other related approaches) lack context-dependence, often making these simple approaches insufficient for the task. What is normal for any given sensor depends on the system's operating mode, time of day, activities in progress, and environmental factors. Simple thresholds that don't take such context into account either end up being so loose that they miss legitimate anomalies, or so tight that too many excess alarms are generated during normal conditions.

In this webinar, I'll show an expert system I've developed in Analytica that detects anomalies and developing failures in our deployed cooling system products. Data from dozens of sensors is collected in 5 minute intervals and the system transitions through multiple operating modes, daily and seasonal environmental fluctuations, and system demands. The Analytica model provides a framework in which complex rules that take multiple factors into account can be expressed, and used to estimate acceptable upper and lower operating ranges that are dynamically adjusted across each moment in time, taking into account whatever context is available. The Analytica environment presents a very readable and understandable language for expressing monitoring rules, and the overall transparency enables us to spot where other rules are needed and what they need to be.

[[Image:AutoMonitoring.png|frame|none|Graph illustrates how upper and lower bounds on operating range is adjusted to context. Actual sensor data is green, the red and blue lines show the computed bounds on acceptable operating range at each point in time.]]

A recording of this webinar can be watched at [http://AnalyticaOnline.com/WebinarArchive/2009-02-05-Failure-detection.wmv Failure-Detection.wmv].

=== Data Center Capacity Planning ===

''Please note that this presentation will be on Wednesday rather than Thursday this week.''

'''Date and Time:''' Wednesday, October 21, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

Data center energy demands are on the rise, creating serious financial as well as infrastructural challenges for data center operators. In 2006, data centers were responsible for a costly 1.5 percent of total U.S. electricity consumption, and national energy consumption by data centers is expected to nearly double by 2011. For data center operators, this means that many data centers are reaching the limits of power capacity for which they were originally designed. In fact, Gartner predicts that 50 percent of data centers will discover they have insufficient power and cooling capacity in 2008.

This week's presentation will provide an overview of [http://www.lumina.com/ADCAPT/Analytica%20Data%20Center%20Capacity%20Planning%20Tool%20Overview.pdf ADCAPT -- the Analytica Data Center Capacity Planning Tool]. For this webinar, the User Group will be joining a presentation that is also being given outside of the Analytica User Group, but I (Lonnie) think is also of interest to the User Group community in that it shows of an example of a re-usable Analytica model, containing several very interesting and novel techniques, applied to a very interesting application area.

Due to technical difficulties, this webinar was not recorded.

=== Modeling the Precision Strike Process ===

'''Date and Time:''' Thursday, October 16, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Henry Neimeier, MITRE

'''Abstract'''

We describe a new paradigm for modeling, and apply it to a simple view of the precision strike attack process against mobile targets. The new modeling paradigm employs analytic approximation techniques that allow rapid model development and execution. These also provide a simple dynamic analytic risk evaluation capability for the first time. The beta distribution is used to summarize a broad range of target dwell and execution time scenarios in compact form. The data processing and command and control processes are modeled as analytic queues.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2008-10-16-Precision-Strike-Process.wmv Precision-Strike-Process.wmv]. Several related papers and materials are also available, including:
* [[media:MILCOM96.pdf|A New Paradigm For Modeling The Precision Strike Process (U)]] by H. Neimeier from MILCOM96
* [[media:Milcom96.ana|Milcom96.ana]] -- the model from the talk and above paper.
* [[media:PhalanxMar.pdf|Analytic Uncertainty Modeling Versus Discrete Event Simulation]] by H. Neimeier, PHALANX March 1996.
* [[media:AnalQueNet.pdf|Analytica Queuing Networks]] by H. Neimeier, Proc. 12th Int'l Conf. Systems Dynamics Soc. 1994.
* [[media:Kuskey CAPE MTR-11.pdf|The Architecture of CAPE Models]] by K.P. Kuskey and S.K. Parker, MITRE Tech. Report.
* [[media:Cmac2pap_2_.pdf|Analytical Modeling in Support of C4ISR Mission Assessment (CMA)]] by F.R. Richards, H.A. Neimeier, W.L. Hamm, and D.L. Alexander, 3rd Int'l Symp. on Command and Control Research and Technology, 1997.
* [[media:INCOSE96.pdf|Analyzing Processes with HANQ]] by H. Neimeier and C. McGowan, MITRE, from INCOSE96.
* Functions for drawing ''radar plots'': [[media:Radarplt.ANA|Radarplt.ana]]
* Power point slides: [[media:Cape.ppt|Cape.ppt]], [[media:PGMrisk.ppt|PGMrisk.ppt]], and [[media:JDEMweb.ppt|JDEMweb.ppt]]

=== Modeling Utility Tariffs in Analytica ===

Presenter: Brian Parsonnet, Ice Energy

Date and Time: Thursday, Nov 8, 2007 at 10:00 - 11:00am Pacific Standard Time

Modeling utility tariffs is a tedious and complicated task. There is no standard approach to how a utility tariff is constructed, and there are 1000’s of tariffs in the U.S. alone. Ice Energy has made numerous passes at finding a “simple” approach to enable tariff vs. product analysis, including writing VB applications, involved Excel spreadsheets, using 3rd party tools, or outsourcing projects to consultants. The difficulty stems from the fact that there is little common structure to tariffs, and efforts to standardize on what structure does exist is confounded by an endless list of exceptions. But using the relatively simple features of Analytica we have created a truly generic model that allows a tariff to be defined and integrated in just a few minutes. The technique is not fancy by Analytica standards, so this in essence demonstrates how Analytic’s novel modeling concept can tackle tough problems.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-08-Tariff-Modeling.wmv 2007-11-08-Tariff-Modeling] (Requires Windows Media Player)

=== Modeling Energy Efficiency in Large Data Centers ===

Date and TimeThursday, Oct 25, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter:Surya Swamy, Lumina Decision Systems

Abstract

The U.S. data center industry is witnessing a tremendous growth period stimulated by increasing demand for data processing and storage. This has resulted in a number of important implications including increased energy costs for business and government, increased emissions from electricity generation, increased strain on the power grid and rising capital costs for data center capacity expansion. In this webinar, Analytica's dynamic modeling capabilities coupled with it's advanced uncertainty capabilities, which offer tremendous support in building cost models for planning and development of energy efficient data centers, will be illustrated. The model enables users to explore future technologies, the performance, costs and efficiencies of which are uncertain and hence to be probabilistically evaluated over time.

You can watch a recording of this presentation at: [http://AnalyticaOnline.com/WebinarArchive/2007-10-25-Data-Center-Model.wmv Data-Center-Model.wmv] (Requires Windows Media Player)

= Graphing =

=== Creating Scatter Plots ===

'''Date and Time:''' Thursday, May 15, 2008 at 10:00 - 11:00am Pacific Daylight
Time

'''Date and Time:''' Thursday, Aug 23, 2007 at 10:00 - 11:00am Pacific Daylight
Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This webinar focuses on utilizing graphing functionality new to Analytica 4.0, and specifically, functionality enabling the creative use of scatter plots. The talk will focus primarily on techniques for simultaneously displaying many quantities on a single 2-D graph. I'll discuss several methods in which multiple data sources (i.e., variable results) can be brought together for display in a single graph, including the use of result comparison, comparison indexes, and external variables. I'll describe the basic new graphing-role / filler-dimension structure for advanced graphing in Analytica 4.0, enabling multiple dimensions to be displayed on the horizontal and vertical axes, or as symbol shape, color, or symbol size, and how all these can be rapidly pivoted to quickly explore the underlying data. I'll discuss how graph settings adapt to changes in pivot or result view (such as Mean, Pdf, Sample views).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-05-15-Scatter-Plots.wmv Scatter-Plots.wmv].

Model used: During this webinar, I started with some example data in the model [[media:Chemical elements orig.ANA|Chemical elements.ana]]. The original file is in the form before graph settings were changed. By the end of the webinar, many graph settings had been altered, and various changes made, resulting in [[media:UG_Webinar_Scatter_Plots.ana|Scatter-Plots.ana]] (during the Aug 23 presentation, this was the final model: [[media:Chemical elements2.ANA|Chemical elements2.ana]]).

=== Graph Style Templates ===

''' Date and Time:''' Thursday, February 28, 2008, 10:00 - 11:00 Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Graph style templates provide a convenient and versitile way to bundle graph setup options so that they can be reused when viewing other result graphs. For example, if you've discovered a set of colors and fonts and a layout that creates the perfect pizzazz for your results, you can bundle that into a template where you can quickly select it for any graph. In this talk, I'll introduce how templates can be used and how you can create and re-use your own. I'll show the basics of using existing templates, previewing what templates will look like, and applying a given template to a single result or to your entire model. We'll also see how to create your own templates, and in the process I'll discuss what settings can be controlled from within a template. I'll discuss how graph setup options are a combination of global settings, template settings, and graph-specific overrides. I'll show how to place templates into libraries (thus allowing you to have template libraries that can be readily re-used in different models), and even show how to control a few settings using templates that aren't selectable from the Graph Setup UI. I'll also touch on how different graph setting are associated with different aspects of a graph, ultimately determining how the graph adapts to changes in uncertainty view or pivots.

The model created during this webinar is here: [[media:Graph style templates.ana|Graph style templates.ana]].
You can watch a recording of the webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-02-28-Graph-Style-Templates.wmv Graph-Style-Templates.wmv].

= Scripting =

=== Button Scripting ===

Date and Time: Thursday, Sept. 6, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Max Henrion, Lumina Decision Systems

Abstract

This webinar is an introduction to Analytica's typescript and button scripting. Unlike variable definitions, button scripts can have side-effects, and this can be useful in many circumstances. I'll cover the syntax of typescript (and button scripts), and how scripts can be used from buttons, picture nodes or choice inputs. I'll introduce some of the Analytica scripting language to those who may have seen or used it before. And we'll examine some ways in which button scripting can be used.

You can watch the recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-06-Button-Scripting.wmv Button Scripting.wmv] (Requires Windows Media Player or equiv). The model files and libraries used during the webinar are in [[media:Ana_tech_webinar_on_scripting.zip|Ana_tech_webinar_on_scripting.zip]].

= Analytica User Community =

=== The Analytica Wiki, and How to Contribute ===

'''Date and Time:''' (tentative) Thursday, October 30, 2008, Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The Analytica Wiki is a central repository of resources for active Analytica users. What's more, you -- as an active Analytica user -- can contribute to it. As an Analytica community, we have a lot to learn from each other, and the Analytica Wiki provides one very nice forum for doing so. You can contribute example models and libraries, hints and tricks, and descriptions of new techniques. You can fix errors in the Wiki documentation if you spot them, or add to the information that is there when you find subtleties that are not fully described. If you spend a lot of time debugging a problem, after solving it you could document the issue and how it was solved for your own benefit in the future, as well as for others in the user community who may encounter the same problem. When you publish a relevant paper, I hope you will add it to the page listing publications that utilize Analytica models.

I will provide a quick tour of the Analytica Wiki as it exists today. I'll then provide a tutorial on contributing to the Wiki -- e.g., the basics of how to edit or add content. The [http://Wikipedia.org Wikipedia] has had tremendous success with this community content contribution model, and I hope that after this introduction many of you will feel more comfortable contributing to the Wiki as you make use of it.

Due to a problem with the audio on the recording, the recording of this webinar is not available.

= Licensing or Installation =

=== Reprise License Manager Tutorial ===

'''Date and Time:''' Wednesday, 11 March 2010, 10:00am Pacific Standard Time

'''Presenter:''' Bob Mearns, Reprise Software Inc.

'''Abstract'''

The Reprise License Manager (RLM) allows all Analytica and ADE licenses within an organization to be managed from a central server. RLM can be used with either ''floating'' or ''named-user'' licenses.

This tutorial on RLM administration is being given by Bob Mearns, lead software developer at Reprise Software, Inc., who has over 15 years' experience developing and supporting software license managers. This session will focus on:

* Basic RLM Server Setup
* How and where RLM looks for licenses
* Using the RLM Web Server Admin Interface
* Using RLM diagnostics, new in RLM v8
* A systematic approach to diagnosing license server connectivity problems

There is a big focus in this talk on how to debug problems with the RLM license manager, and in the process many of the technical details pertaining to the RLM setup are covered. This talk is most relevant for IT managers who administer the license server, and for people who may be installing the RLM server who would like a more thorough understanding of how things work. The RLM license manager is used to host centrally managed licenses, which includes floating and named-user licenses.

This talk is being provided by Reprise Software.

This webinar may be viewed here: [http://analyticaonline.com/WebinarArchive/2010-03-11-RLM-troubleshooting.wmv RLM-troubleshooting.wmv]. The trouble-shooting tips document covered in the talk is at [http://reprisesoftware.com/RLM_Troubleshooting_Tips.pdf RLM Troubleshooting Tips].

See also:
* [[Configuring an RLM Server]] -- step-by-step for installing the RLM server
* [[How to Install Analytica 4.2 -- Centrally Managed License]] -- for the Analytica user's side installation

Tutorial videos

2011-02-25T00:49:50Z

Psanford: /* Introduction to Structured Optimization */

New: I've attempted to impose some categorization on the past user group webinar topics.

The most recent talks were:
* [[#Introduction_to_Structured_Optimization|Introduction to Structured Optimization]] (24 Feb 2011)
* [[#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]] (Session 1: 29 Apr 2010, Session 2:6 May 2010, Session 3:13 May 2010, Session 4:20 May 2010, Session 5:3 June 2010), Session 6:10 June 2010, Session 7:24 June 2010, Session 8:15 July 2010.
* [[#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[#Spearman_Rank_Correlation|Spearman Rank Correlation]] (25 Mar 2010)

= Table and Array Topics =

=== The Basics of Analytica Arrays and Indexes ===

''This webinar is continued across two sessions.''

Date and Time (Part 1): January 10, 2008, 10:00 - 11:00 Pacific Standard Time

Date and Time (Part 1, repeat): January 11, 2008, 10:00 - 11:00 Pacific Standard Time

Date and Time (Part 2): January 17, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

This introductory talk introduces the basic concepts of Analytica indexes and multi-dimensional arrays, as well as the basics of Intelligent Array Abstraction. There are several important differences between Analytica arrays compared to multi-dimensional arrays found in other modeling, database, and programming environments. For example, each dimension of an array is associated with an index object, and there is no inherent ordering to the dimensions of a multi-D array. Intelligent array abstraction is perhaps the most powerful feature in Analytica. The session will include a brief description of what array abstraction does, and how you should take advantage of it.

Part 1 focuses on indexes, 1-D arrays and the uses of the [[Subscript/Slice Operator]].

Part 2 focuses on [[:Category:Array Functions|array functions]], multi-D arrays, and the principles and philosphy of arrays in Analytica.

This talk is intended for beginning Analytica modelers, and for people who have been using Analytica without making substantial use of its array features.

A recording of the two sessions can be viewed at (requires Windows Media Player):
* [http://AnalyticaOnline.com/WebinarArchive/2008-01-11-Intro-to-arrays.wmv Intro-to-arrays (Part 1).wmv]
* [http://AnalyticaOnline.com/WebinarArchive/2008-01-17-Intro-to-arrays2.wmv Intro-to-arrays (Part 2).wmv]

An Analytica model containing the examples created during the webinar can be downloaded from [[media:Intro to intelligent arrays.ana|Intro to intelligent arrays.ana]]. During part 1, the [[media:Plane catching decision with EVIU.ana|Plane catching decision with EVIU.ana]] was also used briefly during the webinar.

=== Manipulating Indexes and Arrays in Analytica Expressions ===

Date and Time: Thursday, Aug 9, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this webinar, I will review many of the common operations applied to indexes and arrays from within Analytica expressions, with a particular emphasis on enhancements in this area that are new to Analytica 4.0. I'll review the often used and very powerful [[Subscript]] and [[Slice]] operations, along with the [[Associative_vs._Positional_Indexing|duality of associational and positional indexing]]. I'll introduce newly introduced extensions for positional indexes, such as the @I, A[@I=n], and @[I=n] operations, and extensions that expose positional duals to various previously-existing associational array functions. I will describe the distinction between index and value contexts in Analytica expressions, along with the distinction between a variable's index value, mid value and sample value, how these may differ ([[Self-Indexed Arrays]]), and how we may access each context-value explicitly. I will also introduce slice assignment -- the ability to assign values to individual slices of an array within an algorithm.

The content of this webinar is most appropriate for users with moderate to advanced Analyica model-building experience.

Here is the Analytica model that was created during this talk: [[media:Indexes and Arrays UG2.ANA | "Indexes and Arrays UG2.ANA"]]. (This wouldn't be very interesting for someone who didn't attend, but it contains the examples we tried).

=== Local Variables ===

'''Date and Time:''' Thursday, 23 July 2009, 10:00-11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

I'll explain distinctions between different types of local variables that can be used within expressions. These distinctions are of primary interest for people implementing [[Meta-Inference]] algorithms, since they have a lot to do with how [[Handle]]s are treated. Analytica 4.2 introduces some new distinctions to the types of local variables, designed to make the behavior of local variables cleaner and more understandable. One type of local variable is the [[LocalAlias]], in which the local variable identifier serves as an alias to another existing object. In contrast, there is the [[MetaVar]], which may hold a [[Handle]] to another object, but does not act as an alias. The only local variable option that existed previously, declared using [[Var..Do]], is a hybrid of these two, which leads to confusion when manipulating [[handle]]s. Since [[LocalAlias..Do]] and [[MetaVar..Do]] have very clean semantics, the use of these when writing [[Meta-Inference]] algorithm should help to reduce that confusion considerably. Inside a [[User-Defined Function]], parameters are also instances of local variables, and depending on how they are declared, may behave as a [[MetaVar]] or [[LocalAlias]], so I'll discuss how these fit into the picture, as well as local indexes and local indexes.

This is appropriate for advanced Analytica modelers.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-07-23-Local-Variables.wmv Local-Variables.wmv]. The analytica file from the webinar is at [[media:Local Variables.ana|Local Variables.ana]], where I've also implemented the exercises that I had suggested at the end of the webinar, so you can look in the model for the solutions.

=== Array Concatenation ===

'''Date and Time:''' Thursday, 25 June 2009 10:00am-11:00 Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Array concatenation combines two (or more) arrays by joining them side-by-side, creating an array having all the elements of both arrays. The special case of list-concatenation joins 1-D arrays or lists to create a list of elements that can function as an index. Array concatenation is a basic, and common, form of array manipulation.

The [[Concat]] function has been improved in Analytica 4.2, so that array concatenation is quite a bit easier in many cases, and the [[ConcatRows]] function is now built-in (formerly it was available as a library function).

I'll take you through examples of array concatenation, including cases that have been simplified with the 4.2 enhancements, to help develop your skills at using [[Concat]] and [[ConcatRows]].

This webinar is appropriate for all levels of Analytica modelers.

You can view a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-06-25-Array-Concatenation.wmv Array_Concatenation.wmv]. The model file created during the webinar is: [[media:Array_Concatenation.ana|Array_Concatenation.ana]].

=== Flattening and Unflattening of Arrays ===

Date and Time: January 31, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

On occassion you may need to flatten a multi-dimensional array into a 2-D table. The table could be called a ''relational representation'' of the data. In some circles it is also refered to as a ''fact table''. Or, you may need to convert in the other direction -- expanding, or unflattening a relational/fact table into a multi-dimensional array. In Analytica, the [[MdTable]] and [[MdArrayToTable]] functions are the primary tools for unflattening and flattening. In this session, I'll introduce these functions and how to use them, several examples, and many variations.

The model developed during this talk is at [[media:Flattening_and_Unflatting_Arrays.ana | Flattening_and_Unflatting_Arrays.ana]]. A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-01-31-Array-Flattening.wmv Array-Flattening.wmv]

=== The [[Aggregate]] Function ===

'''Date and Time:''' Thursday, 2 July 2009, 10:00am - 11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Aggregation is the process of transforming an array based on a fine-grain index into a smaller array based on a coarser-grain index. For example, you might map a daily cash stream into monthly revenue (i.e., reindexing from days to months).

This has always been a pretty common operation in Analytica models, with a variety of techniques for accomplishing it, but it has just become more convenient with the [[Aggregate]] function, new to Analytica 4.2.

In the webinar, I'll be demonstrating the use and generality of the [[Aggregate]] function. In the process, it will also be a chance to review a number of other basic intelligent array concepts, including array abstraction, subscripting, re-indexing, etc.

This webinar is appropriate for all levels of Analytica modelers.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-07-02-Aggregate.wmv Aggregate.wmv]. The model file created during this webinar is: [[media:Aggregate_Function.ana|Aggregate Function.ana]].

=== Sorting ===

'''Date and Time:''' Thursday, 6 Aug 2009, 10:00am-11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This webinar will demonstrate the functions in Analytica that are used to sort (i.e. re-order) data -- the functions [[sortIndex]], [[Rank]], and the new to 4.2 [[Sort]]. I'll cover the basics of using these functions, including how they interact with indexes, how to apply them to arrays of data, and their use with array abstraction. I'll then introduce several new 4.2 extensions for handling multi-key sorts, descending options, and case insensitivity.

This webinar is appropriate for all levels of Analytica modelers.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-08-06-Sorting.wmv Sorting.wmv]. The model file created during the webinar is at [[media:Sorting.ana|Sorting.ana]].

=== [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]] ===

Date and Time: January 24, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

Every dimension of an Analytica array is associated with an index object. [[Array Abstraction]] recognizes when two arrays passed as parametes to an operator or function contain the same indexes. These indexes are more commonly defined by a global index object, i.e., an index object that appears on a diagram as a parallelogram node. However, variable and decision nodes can serve as indexes, and can even have a multi-dimensional value in addition to being an index itself. This is refered to as a [[Self-Indexed Arrays|self index]]. If a variable identifier is used in an expression, the context in which it appears always makes it clear whether the identifier is being used as an index, or as a variable with a value. Self-indexes can arise in several ways, which I will cover. In rare cases, when writing an expression, you may need to be aware of whether you intend to use the index value or the context value of a self-indexed variable. I'll discuss these cases, for example in [[For..Do]] loops, and the use of the [[IndexValue]] function.

In some cases, lists may be used in expressions, and when combined with other results, lists can end up serving as an [[Implicit Dimensions|implicit dimension]] of an array. An implicit dimension is a bit different from a full-fledged index since it has not name, and hence no way to refer to it in an expression where an index parameter is expected. Yet most built-in Analytica functions can still be employed to operate over an implicit index. When an implicit index reaches the top level of an expression, it is promoted to be a self-index. I will explain and demonstrate these concepts.

The model developed during this talk is at [[media:Self-Indexes_Lists_and_Implicit_dimensions.ana|Self-Indexes_Lists_and_Implicit_dimensions.ana]]. A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-01-24-Self-Indexes-Implicit-Dims.wmv Self-Indexes-Implicit-Dims.wmv]

=== Introduction to [[DetermTable]]s ===

'''Date and Time:''' 18 September 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A [[DetermTable]] provides an input view like that of an [[Table|edit table]], allowing you to specify values or expressions in each cell for all index combinations; however, unlike a [[Table|table]], the evaluation of a [[DetermTable|determtable]] conditionally returns only selected values from the table. It is called a determtable because it acts as a deterministic function of one or more discrete-valued variables.
You can conceptualize a determtable as a multi-dimensional generalization of a select-case statement found in many programming languages, or as a value that varies with the path down a decision tree.

[[DetermTable]]s can be used to encode a table of utilities (or costs) for each outcome in a probabilistic model. In this usage, they combine very naturally with [[ProbTable]]s (probability tables) for discrete probabilistic models. They are also extremely useful in combination with [[Choice]] pulldowns, allowing you to keep lots of data in your model, but using only a selected part of that for your analysis. This leads to [[Selective Parametric Analysis]], which is often an effective way of coping with memory capacity limitation in high dimensional models.

In this talk, I'll introduce the [[DetermTable]], show how you create one and describe the requirements for the table indexes. The actual "selection" of slices occurs in the table indexes. Not all indexes have to be selectors, but I'll explain the difference and how the domain attribute is used to establish the table index, while the value is used to select the slice. When you define the domain of a variable that will serve as a [[DetermTable]] index, you have the option of defining the domain as an ''index domain''. This can be extremely useful in combination with a [[DetermTable]], so I will cover that feature as well. It is helpful to understand how the functionality a [[DetermTable]] can be replicated using two nodes -- the first containing an [[Table|Edit Table]] and the second using [[Subscript]]. Despite this equivalence, [[DetermTable]] can be especially convenient, both because it simplifies things by requiring one less node, but also because an [[Table|Edit Table]] can be easily converted into a [[DetermTable]].

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-09-18-DetermTables.wmv DetermTables.wmv]. The examples created while demonstrating the mechanics of [[DetermTable]]s is saved here: [[media:DetermTable intro.ana|DetermTable intro.ana]]. Other example models used were the ''2-branch party problem.ana'' and the ''Compression post load calculator.ana'', both distributed in the Example models folder with Analytica, and the [[media:Loan_policy_selection.ANA|Loan policy selection.ana]] model.

=== [[Table Splicing]] ===

'''Date and Time''': Thursday, August 14, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Edit tables, probability tables and determ tables automatically adjust when their index's values are altered. When new elements are inserted into an index, rows (or columns or slices) are automatically inserted, and when elements are deleted, rows (or columns or slices) are deleted from the tables. This process of adjusting tables is referred to as ''[[Table Splicing|splicing]]''.

Some indexes in Analytica may be computed, so that changes to some input variables could result in dramatic changes to the index value, both in terms of the elements that appear and the order of the elements in the index. This creates a correspondence problem for Analytica -- how do the rows after the change correspond to the rows before the change. Analytica can utilize three different methods for determining the correspondence: associative, positional, or flexible correspondence. I'll discuss what these are and show you how you can control which method is used for each index.

When slices (rows or columns) are inserted in a table, Analytica will usually insert 0 (zero) as the default value for the new cells. It is possible, however, to explicit set a default value, and even to set a different default for each column of the table. Doing so requires some typescripting, but I'll take you through the steps.

Using blank cells as a default value, rather than zero, has some advantages. It becomes quickly apparent which cells need to be filled in after index items are inserted, and Analytica will issue a warning message if blank cells exist that you haven't yet filled in. I'll take you through the steps of enabling blank cells by default.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-08-14-Edit-Table-Splicing.wmv Edit-Table-Splicing.wmv]. (Note: There is a gap in the recording's audio from 18:43-27:35).

=== [[StepInterp|Step Interpolation]] ===

'''Date and Time:''' Thursday, 8 April 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[StepInterp]] function is useful in a number of scenarios, including:
* Discretizing a continuous quantity into a set of finite buckets.
* Looking up a value from a "schedule table" (e.g., tax-rate table, depreciation table)
* Mapping from a date to its fiscal year, when the fiscal year starts on an arbitrary mid-year date.
* Mapping from a cumulated value back to the index element/position.
* Performing a "nearest" or "robust" [[Subscript]] or [[Slice]] operation.
* Interpolating value for a relationship that changes in discrete steps

In this webinar, I'll demonstrate how to use the [[StepInterp]] function on several simple examples.

This webinar is appropriate for beginning Analytica modelers and up.

You can watch a recording of this webinar at: [http://analyticaonline.com/WebinarArchive/2010-04-08-StepInterp.wmv StepInterp.wmv]. You can download the model created during this webinar from [[media:Step Interp Intro.ana|Step Interp Intro.ana]].

=== Analytica Web Player ===

'''Date and Time:''' Thursday, August 7, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

The Analytica Web Player (AWP) is scheduled to launch on July 31, 2008. AWP is a subscription service hosted on Lumina's servers. As a subscriber, you can upload your models to the server and send your colleagues a URL so that they can view your model. To view your models, they need only a Flash-enabled web browser. They can browser your model, change inputs, and evaluate results, all from within their web browser.

In this talk we'll cover the available subscription plans, pricing, limitations, and how you sign up. We'll also demonstrate the process of uploading models and sharing these with colleagues.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-08-07-AWP.wmv AWP.wmv].

=== [[SubTable|SubTables]] ===

'''Date and Time:''' Thursday, 31 July 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[SubTable]] function allows a subset of another edit table to be edited by the user as a different view. To the user, it appears as if he is editing any other edit table; however, the changes are stored in the original edit table. The rows and columns can be transformed to other dimensions in the Subtable, with different index element orders, based on [[Subset]] indexes, and with different number formats.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-31-SubTables.wmv SubTables.wmv]. The model file from this webinar is at [[media:SubTable_webinar.ana]].

=== Edit Table Enhancements in Analytica 4.0 ===

Date and Time: Thursday, Aug 2, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this webinar, I will demonstrate several new edit table functionalities in Analytica 4.0, including:
* Insert [[Choice]] drop-down controls in table cells.
* [[Table Splicing|Splicing]] tables based on computed indexes.
* Customizing the [[TableCellDefault|default cell value(s)]].
* Blank cells to catch entries that need to be filled in.
* [[SubTable]]s
* Using different number formats for each column.

This talk is oriented for model builders with Analytica model-building experience.

The Analytica session that existed by the end of the talk is stored in the following model file: [[Media:Edit_Table_Features.ANA | "Edit Table Features.ana"]].

= Modeling Time =

=== Manipulating Dates in Analytica ===

Date and Time: Thursday, Sept. 13, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this talk, I'll cover numerous aspects relating to the manipulation of dates in Analytica. I'll introduce the encoding of dates as integers and the date origin preference. I'll review how to configure input variables, edit tables, or even individual columns of edit tables to accept (and parse) dates as input. I'll cover date number format capabilities in depth, including how to create your own custom date formats, understanding how date formats interact with your computer's regional settings, and how to restrict a date format to a single column only. We'll also see how axis scaling in graphs is date-aware.

Next, we'll examine various ways to manipulate dates in Analytica expressions. This includes use of the new and powerful functions [[MakeDate]], [[DatePart]], and [[DateAdd]], and some interesting ways in which these can be used, for example, to define date sequences. Finally, we'll practice our array mastery by aggregating results to and from different date granularities, such aggregating from a month sequence to a years, or interpolating from years to months.

The model file resulting by the end of the session is available here: [[media:Manipulating Dates in Analytica.ana|Manipulating Dates in Analytica.ana]].

You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-13-Manipulating-Dates.wmv Manipulating Dates.wmv] (Windows Media Player required) Unfortunately, this one seems to have recorded poorly -- the video size is too small. If you magnify it in your media player, it does become readable. Sorry -- I don't know why it recorded like this.

=== The [[Dynamic]] Function ===

'''Date and Time:''' Thursday, 12 June 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function is used for modeling or simulating changes over time, in which values of variables at time t depend on the values of those variables at earlier time points. Analytica provides a special system index named Time that can be used like any other index, but which also has the additional property that it is used by the [[Dynamic]] function for dynamic simulation.

This webinar is a brief introduction to the use of the Dynamic function and to the creation of dynamic models. I'll cover the basic syntax of the [[Dynamic]] function, as well as various ways in which you can refer to values at earlier time points within an expression. Dynamic models result in influence diagrams that have directed cycles (i.e., where you can start at a node, follow the arrows forward and return to where you started), called dynamic loops. Similar ''cyclic dependencies'' are disallowed in non-dynamic influence diagrams.

During the webinar, we'll loop at several simple examples of Dynamic, oriented especially for those of you with little or no experience with using [[Dynamic]] in models. I'll provide some helpful hints for keeping things straight when building dynamic models. For the more seasoned modelers, I'll also try to fold in a few more detailed tidbits, such as some explanation about how dynamic loops are evaluated, and how variable identifiers are interpreted somewhat differently from within dynamic loops.

The model developed (extension of Fibonacci's rabbit growth model) can be downloaded here: [[media:The Dynamic Function.ana|The Dynamic Function.ana]]. A recording of the webinar can be viewed at
[http://AnalyticaOnline.com/WebinarArchive/2008-06-12-Dynamic-Function.wmv Dynamic-Function.wmv].

=== Modeling Markov Processes in Analytica ===

Date and Time: Thursday, Sept. 20, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Matthew Bingham, Principal Economist, Veritas Economic Consulting

Abstract

The class of mathematical processes characterized by dynamic dependencies between successive random variables is called Markov chains. The rich behavior and wide applicability of Markov chains make them important in a variety of applied mathematical applications including population and demographics, health outcomes, marketing, genetics, and renewable resources. Analytica’s dynamic modeling capabilities, robust array handling, and flexible uncertainty capabilities support sophisticated Markov modeling. In this webinar, a Markov modeling application is demonstrated. The model develops age-structured population simulations using a Leslie matrix structure and dynamic simulation in Analytical.

A recording of this session can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2007-09-20-Markov-Processes.wmv Markov-Processes.wmv] (requires Windows Media Player)

An article about the model presented here: [[media:AnalyticaMarkovtext.pdf|AnalyticaMarkovtext.pdf]]

= Analytica Language Features =

=== [[Local Indexes]] ===

Date and Time: Thursday, Dec. 13, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

A [[Local Indexes|local index]] is an index object created during the evaluation of an expression using either the [[Index..Do]] or [[MetaIndex..Do]] construction. Local indexes may exist only temporarily, being reclaimed when they are no longer used, or they may live on after the evaluation of the expression has completed, as an index of the result. Some operations require the use of local indexes, or otherwise could not be expressed.

In this talk, I'll introduce simple uses of local indexes, covering how they are declared using [[Index..Do]], with several examples. We'll see how to access a local index using the [[Dot operator::A.I|A.I]] operator. I'll discuss the distinctions between local indexes and local variables. I'll show how the name of a local index can be computed dynamically, and I'll briefly cover the [[IndexNames]] and [[IndexesOf]] functions.

The model created during this talk is here: [[media:Webinar_Local_Indexes.ana|Webinar_Local_Indexes.ana]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-12-13-Local-Indexes.wmv Local-Indexes.wmv] (Requires Windows Media Player)

=== [[Handle]]s and [[Meta-Inference]] ===

Date and Time: Thursday, Dec. 6, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

Meta-inference refers to computations that reason about your model itself, or that actually alter your model. For example, if you were to write an expression that counted how many variables are in your model, you would be reasoning about your model. Other examples of meta inference include changing visual appearance of nodes to communicate some property, re-arranging nodes, finding objects with given properties, or even creating a transformed model based on portion of your model's structure.

The ability to implement meta-inferential algorithms in Analytica has been greatly enhanced in Analytica 4.0. The key to implementation of meta-inference is the manipulation of [[Handle]]s to objects (formerly refered to as ''varTerms''). This webinar will provide a very brief introduction to handles and using them from within expressions. I will assume you are pretty familiar with creating models and writing expressions in Analyica, but I will not assume that have previous seen or used Handles. This topic is oriented towards more advanced Analytica users.

The model used/created during this webinar as at: [[media:Handle and MetaInference Webinar.ANA|Handle and MetaInference Webinar.ANA]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-12-06-Handles.wmv Handles.wmv] (Requires Windows Media Player)

=== The [[Iterate]] Function ===

Date and Time: Thursday, Nov. 29, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

With [[Iterate]], you can create a recurrent loop around a large model, which can be useful for iterating until a convergence condition is reached, for example. For complex iterations, where many variables are being updated at each iteration, requires you to structure your model appropriate, bundling and unbundling values within the single iterative loop. With some work, [[Iterate]] can be used to simulate the functionality [[Dynamic]], and thus provides one option when a second [[Time]]-like index is needed (although not nearly as convenient as [[Dynamic]]).

In this session, we'll explore how [[Iterate]] can be used.

Here is the model file developed during the webinar: [[media:Iterate Demonstration.ANA|Iterate Demonstration.ANA]]

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-29-Iterate.wmv Iterate.wmv] (Requires Windows Media Player)

=== The [[Using_References|Reference and Dereference Operators]] ===

Date and Time: Thursday, Nov. 15, 2007 at 10:00 - 11:00am Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract
The reference operators make it possible to represent complex data structures like trees or non-rectangular arrays, bundle heterogenous data into records, maintain arrays of local indexes, and seize control of array abstraction in a variety of scenarios. Using a reference, an array can be made to look like an atomic element to array abstraction, so that arrays of differing dimensionality can be bundled into a single array without an explosion of dimensions. The flexibilities afforded by references are generally for the advanced modeler or programmer, but once mastered, they come in useful fairly often.

Here is the model used during the webinar: [[media:Webinar Reference and Dereference Operators.ANA|Reference and Dereference Operators.ana]].
Near the end of the webinar, I encountered a glitch that I was not able to resolve until after the webinar was over. This has been fixed in the attached model. For an explanation of what was occurring, see:
[[Analytica_User_Group/Reference_Webinar_Glitch]].

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-15-Reference-And-Dereference.wmv Reference-And-Dereference.wmv] (Requires Windows Media Player)

=== Writing [[User-Defined Functions]] ===

Date and Time: Thursday, Sept. 27, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

When you need a specialized function that is not already built into Analytica, never fear -- you can create your own [[User-Defined Functions|User-Defined Function (UDF)]]. Creating UDFs in Analytica is very easy. I'll introduce this convenient capability, and demonstrate how UDFs can be organized into libraries and re-used in other models. I'll also review the libraries of functions that come with Analytica, providing dozens of additional functions.

After this introduction to the basics of UDFs, I'll dive into an in-depth look at [[Function Parameter Qualifiers]]. There is a deep richness to function parameter qualifiers, mastery of which can be used to great benefit. One of the main objectives for a UDF author, and certainly a hallmark of good modeling style, should be to ensure that the function fully array abstracts. Although this usually comes for free with simple algorithms, it is sometimes necessary to worry about this explicitly. I will demonstrate how this objective can often be achieved through appropriate function parameter qualification.

Finally, I will cover how to write a custom distribution function, and how to ensure it works with [[Mid]], [[Sample]] and [[Random]].

This talk is appropriate for Analytica modelers from beginning through expert level. At least some experience building Analytica models and writing Analytica expressions is assumed.

The model created during this webinar, complete with the UDFs written during that webinar, can be downloaded here: [[media:Writing User Defined Functions.ana|Writing User Defined Functions.ana]].

You can watch this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-27-Writing-UDFs.wmv Writing-UDFs.wmv] (Windows Media Player required)

=== Custom Distribution Functions ===

'''Date and Time:''' Thursday, 24 July 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Analytica comes with most of the commonly seen [[:category:Distribution Functions|distributions]] built-in, and many additions [[:category:Distribution Functions|distribution functions]] available in the standard libraries. However, in specific application areas, you may encounter distribution types that aren't already provided, or you may wish to create a variation on an existing distribution based on a different set of parameters. In these cases, you can create your own ''User-Defined Distribution Function'' (UDDF). Once you've created your function, you can utilize it within your model like you would any other distribution function.

User-defined distribution functions are really just instances of [[User-Defined Functions]] (UDFs) that behave in certain special ways. This webinar discusses the various functionalities that a user-defined distribution function should exhibit and various related considerations. Most fundamentally, the defining feature of a UDDF is that it returns a median value when evaluated in Mid mode, but a sample indexed by Run when evaluated from Sample mode. This contrasts with non-distribution functions whose behavior does not depend on the Mid/Sample evaluation mode. Custom distributions are most often implemented in terms of existing distributions (which includes Inverse CDF methods for implementing distributions), so that this property is achieved automatically since the existing distributions already have this property. But in less common cases, UDDFs may treat the two evaluation modes differently.

When you create a UDDF, you may also want to ensure that it works with [[Random]]() to generate a single random variate, and supports the [[Distribution Functions#The Over parameter|Over parameter]] for generating independent distributions. You may also want to create a companion function for computing the density (or probability for discrete distributions) at a point, which may be useful in a number of contexts including, for example, during importance sampling. I'll show you how these features are obtained.

There are several techniques that are often used to implement distribution functions. The two most common, especially in Analytica, are the ''Inverse CDF'' technique and the ''transformation from existing distributions'' method. I'll explain and show examples of both of these. The Inverse CDF is particularly convenient in that it supports all sampling methods (Median Latin Hypercube, Random Latin Hypercube, and Monte Carlo).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-24-Custom-Distribution-Functions.wmv Custom-Distribution-Functions.wmv]. The model file created during the webinar is [[media:Custom Distribution Functions.ana|Custom Distribution Functions.ana]].

=== [[Regular Expressions]] ===

'''Date and Time:''' Thursday, 9 July 2009, 10:00am - 11:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Analytica 4.2 exposes a new and powerful ability to utilize Perl-Compatible regular expressions for text expression analysis. This feature has particular applicability for parsing application when importing data. Long known as the feature that makes Perl and Python popular as data file processing languages, that same power is now readily available within Analytica's [[FindInText]], [[SplitText]], and [[TextReplace]] functions.

This talk will only touch on the regular expression language itself (information on which is readily available elsewhere), but instead focuses on the use of these expressions from the Analytica expressions, especially the extracting of text that matches to subpatterns and finding repeated matches.

One relatively complex example that I plan to work through is the parsing of census population data from datafiles downloaded from the U.S. census web site. The task includes parsing highly variable HTML, as well as multiple CSV files with formatting variations that occur from element to element. These variations, which are typical in many sources of data, demonstrate why the flexibility of regular expressions can be extremely helpful when parsing data files.

Regular expressions themselves are extremely powerful, but when overused, can be very cryptic. So even though it is possible to get carried away with this power, it is good to know how to balance the temptation.

This talk is appropriate for moderate to advanced level modelers.

A recording of this webinar can be watched at [http://AnalyticaOnline.com/WebinarArchive/2009-07-09-Regular-Expressions.wmv Regular-Expressions.wmv]. If you are new to regular expressions, I've included a slides on the regular expression patterns that I made use of in [[media:Regular_Expressions.pps|this power point show]] (these were not shown during the webinar). The model file developed during the webinar is [[media:Regular expressions.ana|Regular expressions.ana]].

=== Using the [[Check Attribute]] to validate inputs and results ===

''' Date and Time:''' Thursday, 17 July 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Check Attribute|check attribute]] provides a way to validate inputs and computed results. When users of your model are entering data, this can provide immediate feedback when they enter values that are out of range or inconsistent. When applied to computed results, it can help catch inconsistencies, which can help reduce error rates and accidental introduction of errors later.

In this talk, I'll demonstrate how to define a check validation for a variable, and how to turn on the check attribute visible so that it is visible in the object window. I'll demonstrate how the failed check alert messages can be customized. And perhaps most interestingly, how the check can be used in edit tables for cell-by-cell validation, so that out-of-range inputs are flagged with a red background, and alert balloons pop-up when out-of-range inputs are entered. Cell-by-cell validation when certain restrictions on the check expression are followed, which I'll discuss.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-07-17-Check-Attribute.wmv Check-Attribute.wmv] (Note: There is audio, but screen is black, for first 50 seconds). The model used during this webinar, with the check attributes inserted, is at [[media:Check attribute -- car costs.ana|Check attribute -- car costs.ana]].

=== The Performance Profiler ===

'''Date and Time:''' October 9, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''Requires Analytica Enterprise''

When you have a model that takes a long time to compute, thrashes in virtual memory, or uses up available memory, the Performance Profiler can tell you where your model is spending its time and how much memory is being consumed by each variable to cache results. It is not uncommon to find that even in a very large model, a small number (e.g., 2 to 5) of variables account for the lion's share of time and memory. With this knowledge, you can focus your attention optimizing the definition of those few variables. On several occassions I've achieved more than 100-fold sped up in computation time on large models using this technique.

The Performance Profiler requires with Analytica Enterprise or Optimizer. I'll demonstrate how to use the profiler with some basic discussions of what is does and does not measure. One neat aspect of the profiler is that you can actually activate it after the fact. In otherwords, even though you haven't adding profiling to your model, if you happen to notice something taking a long time, you can add it in to find out where the time was spent.

Using the Profiler is pretty simple, so I expect this session will be somewhat shorter than usual. The content will be oriented primarily to people who are unfamiliar with the profiler, although I will also try to provide some behind the scenes details and can answer questions about it for

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-10-09-Performance-Profiler.wmv Performance-Profiler.wmv]. The model file containing the first few examples from the webinar can be downloaded from [[media:Simple Performance Profiler Example.ana|Simple Performance Profiler Example.ana]].

= Organizing Models =

=== Modules and Libraries ===

'''Date and Time:''' 10 Dec 2009 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Modules form the basic organizational principle of an Analytica model, allowing models to be structured hierarchically, keeping things simple at every level even in very large complex models. You can use linked modules to store your model across multiple files. This capability enables reuse of libraries and model logic across different models, and allows you to divide your model into separate pieces so that different people can work concurrently on different pieces of the model.

In this talk, I will review many aspects of modules and libraries. We'll see how to use linked modules effectively. I'll cover what the the distinctions are between Modules, Libraries, Models and Forms. I'll demonstrate various considerations when adding modules to existing models -- such as whether you want to import system variables or merge (update) existing objects, and some variations on what is possible there. We'll see how to change modules (or libraries) from being embedded to linked, or vise versa, and how to change the file location for a linked module. When distributing a model consisting of multiple module files, I'll go over directory structure considerations (the relative placement of module files), and also demonstrate how you can store a copy of your model with everything embedded in a single file for easy distribution.

I'll also discuss definition hiding and browse-only locking. By locking individual modules, you can create libraries with hidden and unchangeable logic that can be used in the context of other people's models, keeping your algorithms hidden. Or, you can distribute individual models that are locked as browse only, even in the context of a larger model where the remainder of the model is editable.

I'll talk about using linked modules in the context of a source control system, which is often of interest for projects where multiple people are modifying the same model. I'll also reveal an esoteric feature, the Sys_PreLoadScript attribute, and how this can be used to implement your own licensing and protection of intellectual property.

This webinar is appropriate for all levels of Analytica model builders.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-12-10-Linked-Modules.wmv Linked-Modules.wmv]. The starting model used in the webinar can be downloaded from [[media:Loan_policy_selection_start.ana|Loan_policy_selection_start.ana]], and then you can follow along to introduce and adjust modules as depicted in the recording if you like.

= Uncertainty & Probability Topics =

=== Gentle Introduction to Modeling Uncertainty: Webinar Series ===

'''Date and Time:'''
:'''Session 1:''' Thursday, 29 Apr 2010 10:00am Pacific Daylight Time
:'''Session 2:''' Thursday, 6 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Are you someone who has never built a model containing explicit representions of uncertainty? Did that Statistics 1A class you took a long time ago instill a belief that probability distributions are irrelevant to the type stuff you work on? Are you afraid to start representing uncertainty explicitly because you just don't have the statistics background and don't know much about probability and probability distributions?

If any of these sentiments resonate with you, then this webinar (series) may be for you.

These are interactive webinars. Be prepared to answer some questions, and have Analytica fired up in the background. You are going to use it to compute the answer to a couple exercises! Even if you are watching the recording, be ready to complete the exercises.

This webinar series is most appropriate for:
* Beginning Analytica model builders.
* Users of models that present results with uncertainty.
* Accomplished spreadsheet or Analytica model builders who have not previously incorporated uncertainty.
* People looking to learn the basics of probability for the representation of uncertainty.

==== Session 1: Uncertainty and Probability ====

In the first session discusses different sources and types of uncertainty, probability distributions and how they can be used to represent uncertainty, various different interpretations of probabilities and probability distributions, and reasons why it is valuable to represent uncertainty explicitly in your quantitative models.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2010-04-29-Modeling-Uncertainty1.wmv Modeling-Uncertainty1.wmv]. A copy of the model created by the presenter during the webinar (the scholarship example) can be downloaded from [[media:Modeling uncertainty 1 - princeton scholarship.ana|Modeling uncertainty 1 - princeton scholarship.ana]]. Power point slides can be downloaded from: [[media:Modeling Uncertainty 1.ppt|Modeling Uncertainty 1.ppt]].

==== Session 2: Probability Distributions ====

How do you characterize the amount of uncertainty you have regarding a real-valued quantity? This second session explores this question, and introduces the concepts of average deviation (aka absolute deviation), variance and standard deviation. It then introduces the concept of a ''probability distribution'' and the [[Normal]] and [[LogNormal]] distributions. We examine the [http://en.wikipedia.org/wiki/EVIU expected value of including uncertainty] and do a few modeling exercises that demonstrate how it can be highly misleading, even expensive, to ignore uncertainty.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-06-Prob-Distributions.wmv Prob-Distributions.wmv]. The model build during the webinar can be downloaded from [[media:Probability Distributions Webinar.ana|Probability Distributions Webinar.ana]]. Power point slides are at [[media:Modeling Uncertainty 2.ppt|Modeling Uncertainty 2.ppt]].

==== Session 3: Monte Carlo ====

'''Date and Time:''' Thursday, 13 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

In this third webinar in the "Gentle Introduction to Modeling Uncertainty" series, we will see how a probability distribution can be represented as a set of representative samples, and how this leads to a very general method for propagating uncertainty to computed results. This method is known as Monte Carlo simulation.

Analytica represents uncertainty by storing a representative sample, so we'll be learning about how Analytica actually carries out uncertainty analysis. We explore how all the uncertainty result views in Analytica are created from the sample, and learn various 'tricks' for nice histograms for PDF views in various situations.

We'll learn about the [[Run]] index, and how this places samples across different variables in correspondence. We'll learn about the generality of the Monte Carlo for propagating uncertainty, and also learn what Latin Hypercube sampling is.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-13-Monte-Carlo.wmv Monte-Carlo.wmv]. The power point slides are at: [[media:Monte Carlo Simulation.ppt|Monte Carlo Simulation.ppt]]. Example models created during the webinar include: [[media:Representing Uncertainty - Mining Example.ana|Mining Example.ana]], [[media:Representing Uncertainty - Explicit samples.ana|Explicit samples.ana]]and [[media:Representing Uncertainty 3 - Misc.ana|Representing Uncertainty 3 - Misc.ana]] (product of normals and comparison between Latin Hypercube and Monte Carlo precision).

==== Session 4: Measures of Risk and Utility ====

'''Date and Time:''' Thursday, 20 May 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This fourth webinar in the "Gentle Introduction to Modeling Uncertainty" series will explore concepts and quantitative measures of Risk and Utility. We'll discuss various conceptions and types of risk, and explore topics relevant to model-building that include utility and loss functions, expected value, expected utility, risk neutrality, risk aversion, fractiles and Value-at-risk (VaR).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-05-20-Risk-And-Utility.wmv Risk-And-Utility.wmv]. The power point slides can be viewed at [[media:Measures of Risk and Utility.ppt|Measures of Risk and Utility.ppt]]. There is an interesting modeling exercise and exploration of Expected Shortfall near the end of the power point slides that was not covered during the webinar. The worked out model examples from the webinar, along with a solution to the final example not covered, can be downloaded from [[media:Measures of Risk.ana|Measures of Risk.ana]].

==== Session 5: Risk Analysis for Portfolios ====

'''Date and Time:''' Thursday, 3 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Committing to a single project or investing in a single asset entails a certain amount of risk along with the potential payoff. If you are able to proceed with multiple projects or invest in multiple assets, the degree of risk may be reduced substantially with small impact on potential return. In this fifth webinar in the "Gentle Introduction to Modeling Uncertainty" series, we'll look at modeling portfolios, such as portfolios of investments or portfolios of research and development projects, and the impact this has on risk and return. Portfolio analysis is the basis for practices such as diversification and hedging, and is a key of risk management.

As with other topics in this webinar series, the presentation and discussion is designed for people who are new to the use of these concepts in a model building context.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2010-06-03-Portfolio-Risk.wmv Portfolio-Risk.wmv]. The Power Point slides are at [[media:Risk Analysis for Portfolios.ppt|Risk Analysis for Portfolios.ppt]]. These include some exercises at the end (for homework!) not covered during the webinar, including continuous portfolio allocations. The model developed during the webinar, augmented to include answers to additional exercises is at [[media:Risk Analysis for Portfolios.ana|Risk Analysis for Portfolios.ana]].

==== Session 6: Common Parametric Distributions ====

'''Date and Time:''' Thursday, 10 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

During the first five sessions of the ''Gentle Introduction to Modeling Uncertainty'' webinar series, you have been introduced to three distribution functions: [[Bernoulli]], [[Normal]] and [[LogNormal]]. In this webinar, we're going to increase this repertoire and learn about other common parametric distributions. I'll discuss situations where specific distributions are particularly convenient or natural for expressing uncertainty about certain types of quantities, and other reasons for why you might prefer one particular distribution type over another. We'll also examine the distinction between discrete and continuous distributions.

As with other topics in this webinar series, the presentation and discussion is designed for people who are new to the use of these concepts in a model building context.

A recording of the webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-06-10-Parametric-Distributions.wmv Parametric-Distributions.wmv]. The power point slides are at [[media:Common Parametric Distributions.ppt|Common Parametric Distributions.ppt]], and the Analytica model containing the exercises and solutions to exercises not covered during the live recording is at [[media:Common-Parametric-Distributions.ana|Common-Parametric-Distributions.ana]].

==== Session 7: Expert Assessment of Uncertainty ====

'''Date and Time:''' Thursday, 24 June 2010 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

For most uncertainty analysis, uncertainties about many key quantities must be assessed by expert judgment. There has been a lot of empirical research on human abilities to express their knowledge and uncertainty in the form of probability distributions. It shows that we are liable to a variety of biases, such as overconfidence and motivational biases. I'll give an introduction to practical methods developed by decision analysts to avoid or minimize these biases. I'll give some examples from recent work in expert elicitation for the Department of Energy on the future performance of renewable energy technologies.
I'll also discuss ways to aggregate judgments from different experts.

The session is appropriate for people who are new to this area. This probably includes just about everybody!

This session will draw from Chapters 6 and 7 of "Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis" by M Granger Morgan & Max Henrion, Cambridge University Press, 1992

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2010-06-24-Expert-Assessment.wmv Expert-Assessment.wmv]. (At this time, there seems to be a problem with the uploaded file. We'll attempt to re-upload it, hopefully on Fri 16 July 2010).

==== Session 8: Hypothesis Testing ====

'''Date and Time:''' Thursday, 15 July 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Hypothesis testing from classical statistics addresses the question of whether the apparent support for a given hypothesis is statistically significant. In the field of classical statistics, this is perhaps the most heavily emphasized application of probability concepts, and the methodology is used (if not required by editors) when publishing results for research studies in nearly every field of empirical study.

To illustrate the basic idea, suppose a journalist selects 10 Americans at random and asks whether they support a moratorium on deep sea drilling. Seven of the 10 respond "yes", so the the next day he publishes his article "The Majority of Americans Support a Moratorium on Deep Sea Drilling". His sample is certainly consistent with this hypothesis, but his conclusion is not credible because with such a small sample, this majority could have easily been a random quirk (sampling error). Hence we would say that the conclusion is not "''statistically significant''". But how big does the sample have to be to achieve statistical significance? Where should we draw the line when determining whether the data's support is statistically significant? These are the types of questions addressed by this area of statistics.

Hypothesis testing is a central topic in every introductory Statistics 1A course, often comprising more than half of the total course syllabus. But most introductory courses emphasize a cookbook approach in favor of a conceptual understanding, apparently in the hope of providing people in non-statistical fields step-by-step recipes to follow when they need to publish results in their own fields. As a result, the methodology is possibly misused more often than it is applied correctly, and published results are commonly misinterpreted.

In this seminar, I intend to emphasize a conceptual understanding of the statistical hypothesis methodology rather than the more traditional textbook methodology. After this webinar, when you read "our hypothesis was confirmed by the data at a p-value=0.02 level", or "the hypothesis was rejected with a p-value of 0.18", you should be able to precisely relay what these statements really do or do not imply. You should understand what a p-value and confidence level really denote -- they do not represent, as many people think, the probability that the hypothesis is true.

We will also, of course, examine how we can carry out computations of significance levels (i.e., p-values) within Analytica. Statistics texts are filled with numerous "standard" hypothesis tests (e.g., t-tests, etc), each based on a specific set of assumptions. In this webinar, we'll dive into this in a more general way, where we get to start with our own set of arbitrary assumptions, leveraging the power of Monte Carlo for computation. This means there are no recipes to remember, you can compute significance levels for any statistical model, even if the same assumptions don't appear in your statistics texts, and most importantly, you'll be left with a more general understanding of the concepts.

As a prerequisite, this webinar will assume little more than the introductory background from the earlier webinars in this "Gentle Introduction to Modeling Uncertainty" series. It is appropriate for people who have never taken a Statistics 1A course, or for the majority of people who have taken that introduction to Statistics but could use a refresher.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2010-07-15-Hypothesis-Testing.wmv Hypothesis-Testing.wmv]. To follow along with the webinar, you'll want to also download the Analytica model file [[media:Hypothesis Test S&P Volatility.ana|Hypothesis Test S&P Volatility.ana]] before staring. You'll use the data in that model for the various exercises during the webinar.

Solutions to exercises are saved in this version of the model (created during the webinar): [[media:Hypothesis Test S&P Volatility solution.ana|Hypothesis Test S&P Volatility solution.ana]]. I also inserted a solution to the Parkinson's data test that wasn't covered in the webinar but is contained in the [[media:Hypothesis Testing.ppt|Power Point slides]].

=== Correlated and Multivariate Distributions ===

'''Date and Time:''' Thursday, March 13, 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This talk will discuss various techniques within Analytica for defining probability distributions with specified marginal distributions, and also being correlated with other uncertain variables. Techniques include the use of conditional and hierarchical distributions, multivariate distributions, and Iman-Conover rank-correlated distributions.

The model created during session talk is [[media:Correlated distributions.ana|Correlated distributions.ana]]. You can watch a recording of the webinar from [http://AnalyticaOnline.com/WebinarArchive/2008-03-13-Correlated-Distributions.wmv Correlated-Distributions.wmv].

=== Assessment of Probability Distributions ===

'''Date and Time:''' March 6, 2008 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

''' Abstract'''

When building a quantitative model, we usually need to come up with estimates for many of the parameters and input variables that we use in the model. Because these are estimates, it is good idea to encode these as probability distributions, so that our degree of ''subjective uncertainty'' is explicit in the model. The process of encoding a distribution to reflect the level of knowledge that you (or the experts you work with) have about the true value of the quantity is referred to as ''probability (or uncertainty) assessment'' or ''probability elicitation''.

This webinar will be a highly interactive one, where all attendees are expected to participate in a series of uncertainty assessments as we explore the effects of cognitive biases (such as over-confidence and anchoring), understand what it means to be ''well-calibrated'', and utilize scoring metrics to measure your own degree of calibration. These exercises can help you improve the quality of your distribution assessments, and serve as tools that can help you to when eliciting estimates of uncertainty from other domain experts.

The Analytica model [[media:Probability assessment.ana|Probability assessment.ana]] contains a game of sorts that takes you through several probability assessments and scores your responses. Participants of the webinar played this game by running this model, if you are going to watch the webinar, you will want to do the same. You may want to wait until the appropriate point in the webinar (after preliminary stuff has been covered) before starting. You can watch the webinar recording here: [http://AnalyticaOnline.com/WebinarArchive/2008-03-06-Probability-Assessment.wmv Probability-Assessment.wmv].
The power point slides from the talk are here: [[media:Assessment_of_distributions.ppt|Assessment_of_distributions.ppt]].

=== Statistical Functions ===

'''Date and Time:''' Thursday, 21 Aug 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''This topic was presented in Aug 2007, but not recorded at that time.''

A statistical function is a function that process a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica. I'll describe several built-in statistical functions such as [[Mean]], [[SDeviation]], [[GetFract]], [[Pdf]], [[Cdf]], and [[Covariance]]. I'll demonstrate how all built-in statistical functions can be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index). I'll discuss how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all built-in statistical functions can compute weighted statistics, where each point is assigned a different weight. I'll briefly touch on this feature as a segue into next week's topic, Importance Sampling.

This talk can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-08-21-Statistical-Functions.wmv Statistical-Functions.wmv]. The model built during this talk is available for download at [[media:Intro to Statistical Functions.ana|Intro to Statistical Functions.ana]].

=== [[RankCorrel|Spearman Rank Correlation]] ===

'''Date and Time:''' Thursday, 25 March 2010 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Many measures for quantifying the degree of statistical dependence between quantities are used in statistics. THe two most commonly used are [[Correlation|Pearson's Linear Correlation]] and [[RankCorrel|Spearman's Rank Correlation]], computed respectively in Analytica by the functions [[Correlation]] and [[RankCorrel]]. Pearson's [[Correlation]], which is what people usually mean when they just use the term "Correlation", is a measure of how linear the relationship between two variables is. Spearman's [[RankCorrel|Rank Correlation]] is a measure of monotonic the relationship between two variables is.

This talk provides an introduction to the concept of [[RankCorrel|rank correlation]], how it is distinguished from standard [[Correlation|Pearson correlation]], and what it measures. There are several notable and rather diverse uses of [[RankCorrel]], which include these (and probably many others):
* A quantitative measure of the degree to which two variables are monotonically related. (E.g., the degree to which an increase in one leads to an increase, or decrease, in the other).
* Testing (from measurements) whether two factors are statistically dependent
* Importance analysis: Determining how much the uncertain of an input contributes to the uncertainty of an output.
* Sampling from joint distributions with arbitrary marginals and specified rank-correlations ([[Correlate_With]] and [[Correlate_Dists]])

I will focus mostly on the first two factors in this talk (previous webinars on [[#Sensitivity_Analysis_Topics|Sensitivity Analysis]] have covered the Importance Analysis usage to some extent, and a previous webinar on [[#Correlated_and_Multivariate_Distributions|Correlated and Multivariate Distributions]] has covered the last point).

Standard hypothesis tests exist for determining whether two factors are statistically dependent by testing the hypothesis that their [[RankCorrel|rank correlation]] is non-zero (null hypothesis that it is zero). When the P-value of these tests is less than 5% (or 1%), you would be justified in concluding that the two variables are statistically dependent. I will demonstrate how to compute this P-value.

Then I will introduce a new analysis of [[RankCorrel|rank correlation]] that I came up with, which I think is novel and potentially pretty useful, somewhat related to the classical hypothesis tests just mentioned. Suppose you gather a small sample of data on two variables in a study and you want to determine how strong the monotonicity between the two variables is. You can compute the ''sample rank correlation'' for the data set, but this is only an estimate since you have a small sample size and thus sampling error may throw off this estimate. So suppose we imagine there is some "true" underlying rank correlation between the variables (this in itself is a new concept, which I will make precise). From your data set, you have some knowledge about the true value of this underlying rank correlation -- the larger your sample size, the more precise your knowledge is. The new technique I describe here computes a (posterior) distribution over the true underlying rank correlation, from which you can express your rank correlation result as a range (such as rc=0.6±0.2), and answer questions such as what is the probability that the underlying rank correlation is between -0.1 and 0.1, P(-0.1 < rc < 0.1), or P(rc>0), etc. Although this is essentially a posterior distribution, there is no prior distribution involved or needed to computate it, so it is simply a function of the measured data and of the sample size. It really is a probability distribution on the underlying rank correlation, not just a P-value, making it much more useful.

This new analysis is also useful for quantifying the probability that two factors are independent in a manner not possible with the classical tests. The classical P-value of the aforementioned tests measure the probability of a Type II error for the hypothesis that variables are dependent. These tests do not provide the probability of a Type I error, which would be the criteria for concluding that a claim of statistical independence is statistically signficant. This new measure, however, can justifiably be used for quantifying a claim of statistical independence since it allows P(-c<rc<c) to be computed for any c.

I will demonstrate how this new analysis of rank correlation works and is encoded within Analytica, and show how to read off the interesting results.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2010-03-25-Rank-Correlation.wmv Rank-Correlation.wmv]. The model files created during the talk are available at: [[media:Rank-Correlation-Examples.ana|Rank-Correlation-Examples.ana]] and [[media:Rank-Correlation-Analysis.ana|Rank-Correlation-Analysis.ana]].

=== Statistical Functions in Analytica 4.0 ===

Date and Time: Thursday, Aug 16, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

A statistical function is a function that process a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica 4.0. In Analytica 4.0, all built-in statistical functions can now be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index), eliminating the need for separate function libraries. I will demonstrate this use, as well as several new statistical functions, e.g., [[Pdf]], [[Cdf]], [[Covariance]]. I will explain how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all statistical functions in Analytica 4.0 can compute weighted statistics, where each point is assigned a different weight. I'll cover the basics of sample weighting, and demonstrate some simple examples of using this for computing a Bayesian posterior and for importance sampling from an extreme distribution.

The Analytica model file that had resulted by the end of the presentation can be downloaded here:
[[media:User Group Webinar - Statistical Functions.ANA | User Group Webinar - Statistical Functions.ANA]].

=== The [[Large Sample Library: User Guide|Large Sample Library]] ===

'''Date and Time:''' Thursday, 18 Feb 2010 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Large Sample Library: User Guide|Large Sample Library]] is an Analytica library that lets you run a Monte Carlo simulation for a large model or a large sample size that might otherwise exhaust computer memory, including virtual memory. It breaks up a large sample into a series of batch samples, each small enough to run in memory. For selected variables, known as the ''Large Sample Variables'' or ''LSVs'', it accumulates the batches into a large sample. You can then view the probability distributions for each LSV using the standard methods — confidence bands, PDF, CDF, etc. — with the full precision of the large sample.

Memory is saved by not storing results for non-LSVs.

This presentation introduces this library and how to use it.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2010-02-18-Large-Sample-Library.wmv Large-Sample-Library.wmv]. The Large Sample library can be downloaded for use in your own models from the [[Large Sample Library: User Guide]] page. The two example models used during this webinar were: [[media:Enterprise model3.ANA|Enterprise model3.ana]] and [[media:Simple example for Large Sample Library.ana|Simple example for Large Sample Library.ana]].

= Sensitivity Analysis Topics =

=== Tornado Charts ===

'''Time and Date:''' Thursday, 20 Mar 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract:'''

[[Image:Tornado plot.png]]

A tornado chart depicts the result of a '''local sensitivity analysis''', showing how much a computed result would change if each input were varied one input at a time, with all other inputs held to their baseline value. The result is usually plotted with horizontal bars, sorted with larger bars on top, resulting in a graph resembling the shape of a tornado, hence the name. There a numerous variations on tornado charts, resulting from different ways of varying the inputs, and in some cases, different metrics graphed.

This talk will walk through the steps of setting up a Tornado chart, and explore different variations of varying inputs. We'll also explore some more complex issues that can arise when some inputs are arrays.

The model used during this talk is here: [[media:Tornado Charts.ANA|Tornado Charts.ana]] (the stuff for the talk was in the Tornado Analysis module). You can watch a recording of this webinar from [http://AnalyticaOnline.com/WebinarArchive/2008-03-20-Tornado-Charts.wmv Tornado-Charts.wmv].

=== Advanced Tornado Charts -- when inputs are Array-Valued ===

'''Date and Time:''' Thursday, April 17, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The webinar of 20-Mar-2008 ([http://AnalyticaOnline.com/WebinarArchive/2008-03-20-Tornado-Charts.wmv Tornado-Charts.wmv], see webinar archives) went through the fundamentals of setting up a local sensitivity analysis and plotting the results in the form of a tornado chart. That webinar also discussed the many variations of tornado analyses (or more generally, local sensitivity analyses) that are possible.

This talk builds on those foundations by going a step further and addressing tornado analyses when some of the input variables are array-valued. The presence of array-valued inputs introduces many additional possible variations of analyses, as well as many modeling complications. For example, a local sensitivity analysis varies one input at a time, but that could mean you vary each input variable (as a whole) at a time, or it could mean that you vary each cell of each input array individually. Either is possible, each resulting in a different analysis. Some of these variations compute the correct result automatically through the magic of array abstraction, once you've set up the basic tornado analysis that we covered in the first talk, while other require quite a bit of additional modeling effort. However, even the ones that produce the correct result can often be made more efficient, particularly when the indexes of each input variable are different across input variables.

When we do opt to vary input arrays one cell at a time, the display of the results may be dramatically effected. Although we can keep the results in an array form, the customary tornado chart require us to ''flatten'' the multi-D arrays and label each bar on the chart with a cell coordinate.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-04-17-Tornados-With-Arrays.wmv Tornados-With-Arrays.wmv]. This webinar made use of the following models: [[media:Sales Effectiveness Model with tornado.ana|Sales Effectiveness Model with tornado.ana]], [[media:Biotech R&D Portfolio Tornado.ana|Biotech R&D Portfolio with Tornado.ana]], [[media:Sensitivity Analysis Library.ana|Sensitivity Analysis Library.ana]], and [[media:Sensitivity Functions Examples.ana|Sensitivity Functions Examples.ana]]. See [[The Sensitivity Analysis Library]] for more information on how to use [[media:Sensitivity Analysis Library.ana|Sensitivity Analysis Library.ana]] in your own models.

= Financial Analysis =

=== Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]]) ===

'''Date and Time:''' 18 Dec 2008, 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This is Part 3 of a multi-part webinar series where we have been covering the modeling and evaluation of cash flows over time in an interactive exercise-based webinar format, where concepts are introduced in the form of modeling exercises, and participants are asked to complete the exercises in Analytica during the webinar. Part 3 covers Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]]), and includes seven modeling exercises.

To speed the presentation up, I am providing the exercises in advance: [[media:NPV_and_IRR.ppt|NPV_and_IRR.ppt]]. I urge you to take a shot at completing them before the webinar begins, and we'll advance through the exercises more rapidly so as to complete the topic material within the hour. By attempting the exercises in advance, you'll have a good opportunity to compare your solutions to mine, and to ask questions about things you got stuck on.

A dollar received today is not worth the same as a dollar received next year. Taking this time-value of money (or more generally, time-value of utility) into account is very important when comparing cash flows over time that result from long-term capital budgeting decisions. Net Present Value (NPV) and Internal Rate of Return (IRR) are the two most commonly used metrics examining the effective value of an investment's cash flow over time. Both concepts are pervasive in decision-analytic models.

This webinar will be highly interactive. Fire up a instance of Analtyica as you join on. As I introduce each concept, I'll provide you with cash flow scenarios, and give you a chance to compute the result yourself using Analytica. This talk is intended for people who are not already well-versed in NPV and IRR, or for people who already have a good background with those concepts but are new to Analytica and thus can learn from the interactive practice of addressing these exercises during the talk.

See also the materials from Parts 1 and 2 (Net Present Value, 20 Nov 2008 and 4 Dec 2008) elsewhere on this page. This session begins with the model [[media:Cash Flow Metrics 2.ana|Cash Flow Metrics 2.ana]], and ends with [[media:Cash Flow Metrics 3.ana|Cash Flow Metrics 3.ana]]. You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-12-18-IRR.wmv IRR.wmv].

=== Bond Portfolio Analysis ===

'''Date and Time: ''' 11 Dec 2008, 10:00am Pacific Standard Time

'''Presenter:''' Rob Brown, Incite! Decision Technologies

'''Abstract'''

I demonstrate how to value a bond portfolio in which bonds are bought and sold on an uncertain frequency. The demonstration shows how Intelligent Arrays and related functions can greatly simplify calculations of multiple dimensions that would typically require multiple interconnected sheets in a spreadsheet or nested do-loops in a procedural language.

You can watch a recording of this webinar at
[http://AnalyticaOnline.com/WebinarArchive/2008-12-11-Bond-Portfolio-Analysis.wmv Bond-Portfolio-Analysis.wmv]. The model underlying the presentation is [[media:Bond Portfolio Valuation.ana|Bond Portfolio Valuation.ana]], and the power point slides are at [[media:Bond Portfolio Valuation.ppt|Bond Portfolio Valuation.ppt]].

=== Net Present Value ([[NPV]]) ===

'''Date and Time:''' Part I : Thursday, 20 Nov 2008, 10:00am Pacific Standard Time
::::Part II : Thursday, 4 Dec 2008, 10:00am Pacific Standard Time

(Parts 1 & 2 cover NPV -- part 3, listed now separately, covers IRR)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A dollar received today is not worth the same as a dollar received next year. Taking this time-value of money (or more generally, time-value of utility) into account is very important when comparing cash flows over time that result from long-term capital budgeting decisions. Net Present Value ([[NPV]]) and Internal Rate of Return ([[IRR]]) are the two most commonly used metrics examining the effective value of an investment's cash flow over time. Both concepts are pervasive in decision-analytic models.

This multi-part webinar provides an introduction to the concepts of present value, discount rate, [[NPV]] and [[IRR]]. We'll discuss the interpretation of ''discount rate'', and we'll get practice computing these metrics in Analytica. We'll examine the pitfalls of each metric, and we'll examine the interplay of each metric with explicitly modelled uncertainty (including the concepts of Expected NPV (ENPV) and Expected IRR (EIRR)).

This webinar will be highly interactive. Fire up a instance of Analtyica as you join on. As I introduce each concept, I'll provide you with cash flow scenarios, and give you a chance to compute the result yourself using Analytica. This talk is intended for people who are not already well-versed in [[NPV]] and [[IRR]], or for people who already have a good background with those concepts but are new to Analytica and thus can learn from the interactive practice of addressing these exercises during the talk.

I have assembled quite a bit of material, which I believe will fill two webinar sessions. Part 1 will focus mostly on present value, NPV, discount rate, and the use of NPV with uncertainty. Part 2 will focus mostly on IRR, several "gotchas" with IRR, and MIRR.

Materials:
* [[media:Cash Flow Metrics 2.ana|Cash Flow Metrics 2.ana]] : Model at end of second session
* [[media:Cash Flow Metrics 1.ANA|Cash Flow Metrics 1.ana]] : Model at end of first session
* [http://AnalyticaOnline.com/WebinarArchive/2008-11-20-NPV-and-IRR1.wmv NPV-and-IRR1.wmv] : Webinar recording of Part 1.
* [http://AnalyticaOnline.com/WebinarArchive/2008-12-04-NPV-and-IRR2.wmv NPV-and-IRR2.wmv] : Webinar recording of Part 2.

Note: Part 1 covered 5 exercises, covering present value, discount rate, modeling certain cash flows, computing NPV, and graphing the NPV curve. Part 2 added exercises 6-9, covering cash flows at non-uniformly-spaced time periods, valuating bonds and treasury notes, cash flows with uncertainty, and using the CAPM to find invester-implied corporate discount rate.

The "class" will continue with Part 3 beginning with Internal Rate of Return.

= Data Analysis Techiques =

=== Statistical Functions ===

'''Date and Time:''' Thursday, May 22, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

A statistical function is a function that processes a data set containing many sample points, computing a "statistic" that summarizes the data. Simple examples are [[Mean]] and [[Variance]], but more complex examples may return matrices or tables. In this talk, I'll review statistical functions that are built into Analytica 4.0. In Analytica 4.0, all built-in statistical functions can now be applied to historical data sets over an arbitrary index, as well as to uncertain samples (the Run index), eliminating the need for separate function libraries. I will demonstrate this use, as well as several new statistical functions, e.g., [[Pdf]], [[Cdf]], [[Covariance]]. I will explain how the domain attribute should be utilized to indicate that numeric-valued data is discrete (such as integer counts, for example), and how various statistical functions (e.g., [[Frequency]], [[GetFract]], [[Pdf]], [[Cdf]], etc) make use of this information. In the process, I'll demonstrate numerous examples using these functions, such things as inferring sample covariance or correlation matricies from data, quickly histogramming arbitrary data and using the coordinate index setting to plot it, or using a weighted [[Frequency]] for rapid aggregation.

In addition, all statistical functions in Analytica 4.0 can compute weighted statistics, where each point is assigned a different weight. I'll cover the basics of sample weighting, and demonstrate some simple examples of using this for computing a Bayesian posterior and for importance sampling from an extreme distribution.

This talk is appropriate for moderate to advanced users.

A recording of this webinar can be watched at
[http://AnalyticaOnline.com/WebinarArchive/2008-05-22-Statistical-Functions.wmv Statistical-Functions.wmv].
The model created during this webinar is at [[media:Statistical Functions.ANA|Statistical Functions.ana]].

=== Principal Components Analysis (PCA) ===

'''Date and Time:''' 15 Jan 2009, 10:00am Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Principal component analysis (PCA) is a widely used data analysis technique for dimensionality reduction and identification of underlying common factors. This webinar will provide a gentle introduction to PCA and demonstrate how to compute principal components within Analytica. Intended to be at an introductory level, with no prior experience with PCA (or even knowledge of what it is) assumed.

The model developed during this talk, where the principal components were computed for 17 publically traded stocks based on the previous 2 years of price change data is [[media:Principal Component Analysis.ana|Principal Component Analysis.ana]]. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2009-01-15-PCA.wmv PCA.wmv].

=== Variable Stiffness Cubic Splines ===

'''Date and Time:''' 2 October 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Brian Parsonnet, ICE Energy

'''Abstract'''

The Variable Stiffness Cubic Spline is a highly robust data smoothing and interpolation technique. A stiffness parameter adjusts the variability of the curve. At the extreme of minimal stiffness, the curve approaches a cubic spline (like [[CubicInterp]]) that passes through all data points, while at the other extreme of maximal stiffness, the spline curve becomes the best-fit line. Weight parameters can be used to constrain the curve to include selected points, while smoothing over others. The first, second and third derivatives all exist and are readily available.

I'll introduce and demonstrate [[User-Defined Functions]] that compute the variable stiffness cubic spline and interpolate to new points. I'll also show how these curves can be used to detect or eliminate anomalies in data.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-10-02-Variable-Stiffness-Cubic-Splines.wmv Variable-Stiffness-Cubic-Splines.wmv]. The model and library with the vscs functions will be posted here within a few weeks.

=== Using [[Regression]] ===

'''Date and Time:''' Thursday, May 1, 2008 at 10:00 - 11:00 Pacific Daylight Time

'''Date and Time:''' Thursday, Aug 30, 2007 at 10:00 - 11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Regression analysis is a statistical technique for curve fitting, discovering relationships in data, and testing hypotheses between variables. In this webinar, I will focus on generalized linear regression, which is provided by Analytica's [[Regression]] function, and examine many ways in which is can be used, including fitting simple lines to data, polynomial regression, use of other non-linear terms, and fitting of autoregressive models (e.g., ARMA). I'll examine how we can assess how likely it is the data might have been generated from the particular form of the regression model used. We can also determine the level of uncertainty in our inferred parameter values, and incorporate these uncertainties into a model that uses the result of the regression. The talk will cover Analytica 4.0 functions [[Regression]], [[RegressionDist]], [[RegressionFitProb]], and [[RegressionNoise]].

You can watch a recording of the 1 May 2008 webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-05-01-Regression.wmv Regression.wmv]. The model developed during that webinar is here: [[media:Using_Regression.ANA|Using Regression.ana]]

=== Logistic Regression ===

'''Date and Time:''' Thursday, 5 June 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

''' Abstract'''

''(Features covered in this webinar require Analytica Optimizer)''

Logistic regression is a technique for fitting a model to historical data to predict the probability of an event from a set of independent variables. In this talk, I'll introduce the concept of Logistic regression, explain how it differs from standard [[Regression|linear regression]], and demonstrate how to fit a logistic regression model to data in Analytica. Probit regression is for all practical purposes the same idea as Logistic regression, differing only in the specific functional form for the model. Poisson regression is also similar except is appropriate when predicting a probability distribution over a dependent variable that represents integer "counts". All are examples of generalized linear models, and after reviewing these forms of logistic regression, it should be clear how other generalized linear model forms can be handled within Analytica.

This topic is appropriate for advanced modelers. I will assume familiarity with [[Regression|regression]] (see the earlier talk on the topic), but will not assume a previous knowledge of logistic regression.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2008-06-05-Logistic-Regression.wmv Logistic-Regression.wmv]. The model developed during this webinar can be downloaded from [[media:Logistic_regression_example.ana|Logistic_regression_example.ana]]. You'll also need the file [[media:BreastCancer.data|BreastCancer.data]].

= Bayesian Techniques =

=== Bayesian Posteriors using Importance Sampling ===

'''Date and Time:''' Thursday, September 4, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Several algorithms for computing Bayesian posterior probabilities are special cases of ''importance sampling''. The [[#Importance_Sampling_.28Rare_events.29|webinar of the previous week, Importance Sampling (rare events)]] introduced importance sampling, covered the theory behind it, how it is applied, and how Analytica's sample weighting feature can be use for importance sampling. This webinar continues with importance sampling, this time exploring how it can be used (at least in some cases) to compute Bayesian posterior probabilities.

I'll provide an introduction to what Bayesian posterior probabilities are, describe a couple importance sampling-based approaches to computing them, and implement a few examples in Analytica. Importance sampling techniques for computing posteriors have limited applicability -- in some cases they work well, other not. I'll try to characterize what those conditions are.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-09-04-Posteriors_using_IS.wmv Posteriors_using_IS.wmv]. About two-thirds through the presentation, we noticed a result that seemed to be coming out incorrectly. I explain what the problem was and fix it in [http://AnalyticaOnline.com/WebinarArchive/2008-09-04-Posteriors_using_IS_addendum.wmv Posteriors_using_IS_addendum.wmv]. The models used during this presentation can be downloaded from [[media:Posterior sprinklers.ana|Posterior sprinklers.ana]] and [[media:Likelihood weighting.ana|Likelihood weighting.ana]].

=== Importance Sampling (Rare events) ===

'''Date and Time:''' Thursday, 28 Aug 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Importance sampling is a technique that simulates a target probability distribution of interest by sampling from a different sampling distribution and then re-weighting the sampled points so that computed statistics match those of the target distribution. The technique has has applicability when the target distribution is difficult to sample from directly, but where the probability density function is readily available. The technique produces valid results in the large sample size limit for any selection of sampling distribution (provided it is absolutely continuous with respect to the target distribution), but best results (i.e., fastest convergence with smaller sample size) are obtained when a good sampling distribution is used. The technique is commonly used for rare-event sampling, where you want to ensure greater sampling coverage in the tails of distributions, where few samples would occur with standard Monte Carlo sampling. During the talk, we develop a rare event model. It also has applicability to the computation of Bayesian posteriors, and sampling of complex distribution.

In this talk we cover the theory behind importance sampling and introduce the [[SampleWeighting|sample weighting]] mechanism that is built into Analytica. We develop a rare-event model to demonstrate how the weighting mechanism is used to achieve the importance sampling. Next week we'll continue with an example of computing a Bayesian posterior probability.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-08-28-Importance-Sampling.wmv Importance-Sampling.wmv]. The model developed during this talk can be downloaded from: [[media:Importance sampling rare events.ANA|Importance Sampling rare events.ana]].

= Presenting Models to Others =

=== Guidelines for Model Transparency ===

'''Date and Time:''' 19 Feb 2009, 10:00am Pacific Standard Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

What makes Analytica models easy for others to use and understand? I will review some example models that illustrate ways to improve transparency -- or opacity. Feel free to send me your candidates ahead of time!
We'll review some proposed guidelines. I hope to stimulate a discussion about what you think works well or not, and enlist your help in refining these guidelines.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-02-19-Transparency-Guidelines.wmv Transparency-Guidelines.wmv].

=== Creating Control Panels ===

'''Date and Time:''' Thursday, May 29, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

It is quite easy to put together "control panels" or "forms" for your Analytica models by creating input and output nodes for the inputs and outputs of interest to your model end users. This webinar will cover the basic steps involved in creating and arranging these forms, along with some tricks for making the process efficient. We'll cover the different types of input and output controls that are currently available, the use of text nodes to create visual groupings, use of images and icons, and the alignment commands that make the process very rapid. We'll learn how to change colors, and look at the use of buttons very briefly. This talk is appropriate for beginning Analytica users.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-05-29-Control-Panels.wmv Control-Panels.wmv] (required Windows Media Player). The model used during this webinar is at [[media:Building Control Panels.ana|Building Control Panels.ana]].

=== Sneak preview of Analytica Web Publisher ===

'''Date and Time:''' Thursday, February 21, 2008, 10:00 - 11:00 Pacific Standard Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

In this week's webinar, Max Henrion, Lumina's CEO, will provide a sneak preview of the Analytica Web Publisher. AWP offers a way to make Analytica models easily accessible to anyone with a web browser. Users can open a model, view diagrams and objects, change input variables, and view results as tables and graphs. Users will also be able to save changed models, to revisit them in later sessions. Model builders can upload models into AWP directly from their desktop. Usually, AWP directories are password protected, so only authorized users can view and use models. But, we also plan to make a free AWP directory available for people who want to share their models openly.

AWP is nearing release for alpha testing. We will welcome your comments and hearing how you might envisage using AWP.

''This webinar was not recorded.''

= Application Integration Topics =

=== OLE Linking ===

'''Time and Date:''' Thursday, 27 Mar 2008 10:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract:'''

OLE linking is a commonly used methods for linking data from Excel spreadsheets into Analytica and results from Analytica into Excel spreadsheets. It can be used with other applications that support OLE-linking as well. The basic usage of OLE linking is pretty simple -- it is a lot like copy and paste. This webinar covers basics of using OLE linking of fixed-sized 1-D or 2-D tables. I also demonstrate the basic tricks you must go through to link index values and multi-D inputs and outputs. In addition, we discuss what some of those OLE-link settings actually do, and explain how OLE-connected applications connect to their data sources.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-03-27-OLE-Linking.wmv 2008-03-27-OLE-Linking.wmv].

Note: Another 10 minute fast-paced video (separate from the webinar) demonstrates linking data from Analytica into Excel, computing something from that data, and linking the result back into Analytica: [http://AnalyticaOnline.com/WebinarArchive/OLE-to-Excel-and-back.wmv OLE-to-Excel-and-back.wmv].

=== Querying an OLAP server ===

'''Date and Time:''' Thursday, February 14, 2008, 10:00 - 11:00 Pacific Standard Time 
(''Note: Schedule change from an earlier posting. This is now back to the usual Thursday time. '')

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

In this session, I'll show how the [[MdxQuery]] function can be used to extract multi-dimensional arrays from an On-Line Analytical Processing (OLAP) server. In particular, during this talk we'll query Microsoft Analysis Services using MDX. In this talk, I'll introduce some basics regarding OLAP and Analysis Services, discuss the differences between multi-dimensional arrays in OLAP and Analytica, cover the basics of the MDX query language, show how to form a connection string for [[MdxQuery]], and import data. I'll also show how hierarchical dimensions can be handled once you get your data to Analytica.

''Note: Use of the features demonstrated in this webinar require the Analytica Enterprise or Optimizer edition, or the Analytica Power Player. They are also available in ADE.''

The model created during this webinar is available here: [[media:Using MdxQuery.ana|Using MdxQuery.ana]]. You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-02-14-MdxQuery.wmv MdxQuery.wmv] (requires Microsoft Media Player)

=== Querying an ODBC relational database ===

Date and Time: Thursday, February 7, 2008, 10:00 - 11:00 Pacific Standard Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

In this talk I'll review the basics of querying an external relational ODBC database using [[DbQuery]]. This provides a flexible way to bring in data from SQL Server, Access, Oracle, and mySQL databases, and can also be used to read CSV-text databases and even Excel. In this talk, I will cover the topics of how to configure and specify the data source, the rudimentary basics of using SQL, the use of Analytica's [[DbQuery]], [[DbWrite]], [[DbLabels]] and [[DbTable]] functions.

''Note: Use of the features demonstrated in this webinar require the Analytica Enterprise or Optimizer edition, or the Analytica Power Player. They are available in ADE."

You can grab the model created during this webinar from here: [[media:Querying an ODBC relational database.ANA|Querying an ODBC relational database.ana]]. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-02-07-Using-ODBC-Queries.wmv Using-ODBC-Queries.wmv].

=== Calling External Applications ===

Date and Time: Thursday, Oct 18, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

The [[RunConsoleProcess]] function runs an external program, can exchange data with that program, and can be used to perform a computation or acquire data outside of Analytica, that then can be used within the model. I'll demonstrate how this can be used with a handful of programs, and code written in several programming and scripting languages. I'll demonstrate a user-defined function that retrieves historical stock data from a web site.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-10-18-Calling-External-Applications.wmv Calling-External-Applications.wmv] (Requires Windows Media Player)

Files created or used during this webinar can be downloaded:
* [[media:Regular Expression Matching.ANA|Regular Expression Matching.ana]]
* [[media:RegExp.vbs|RegExp.vbs]]
* [[media:Read Historical Stock Data.ana|Read Historical Stock Data.ana]]
* For plotting to gnuplot, these gnuplot command files were used. (Note: You may have to adjust some file paths within these files, and within the model): [[media:Gnuplot-candlesticks.dat|Gnuplot-candlesticks.dat]], [[media:Gnuplot-3dsurface.dat|Gnuplot-3dsurface.dat]]
* [[media:ReadURL.exe|ReadURL.exe]] (for C++/CLR source code, see [[Retrieving Content From the Web]])

The example of retrieving stock data from Yahoo Finance is also detailed in an article here: [[Retrieving Content From the Web]]

=== New Functions for Reading Directly from an Excel File ===

'''Date and Time:''' Thursday, 24 April 2008 10:00 Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

'''(Feature covered requires Analytica Enterprise or better)'''

Hidden within the new release of Analytica 4.1 are three new functions for reading values directly from Excel spreadsheets: [[OpenExcelFile]], [[WorksheetCell]], [[WorksheetRange]]. These provide an alternative to OLE linking and ODBC for reading data from spreadsheets, which may be more convenient, flexible and reliable in many situations. We have not yet exposed these functions on the Definitions menu or in the Users Guide in release 4.1, since they are still in an experimental stage. I would like know that they have been "beta-tested" in a variety of scenarios before we fully expose them (also, the symmetric functions for writing don't exist yet). In this webinar, I will introduce and demonstrate these functions, after which you can start using them with your own problems.

The model created during this talk is here: [[Media:Functions for Reading Excel Worksheets.ana|Image:Functions for Reading Excel Worksheets.ana]]. It read from the example that comes with Office 2003, to which we added a few range names during the talk, resulting in [[media:SOLVSAMP.XLS|SolvSamp.xls]]. Place the excel file in the same directory as the model. A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-04-24-Reading-From-Excel.wmv Reading-From-Excel.wmv].

=== Reading Data from URLs to a Model ===

'''Date and Time:''' Thursday, 27 Aug 2009, 10:00am-11:00am Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

''Requires Analytica Enterprise''

The new built-in function, [[ReadFromUrl]], can be used to read data (and images) from websites, such as HTTP web pages, FTP pages, or even web services like SOAP. In this webinar, I'll demonstrate the use of this function in several ways, including reading live stock and stock option price data, posting data to a web form, retrieving a text file from an FTP site, supplying user and password credentials for a web site or ftp service, downloading and displaying images including customized map and terrain images, and querying a SOAP web service.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2009-08-27-ReadFromUrl.wmv ReadFromUrl.wmv]. The model with the examples shown during the webinar is at [[media:Reading_Data_From_the_Web.ana|Reading_Data_From_the_Web.ana]].

=== Using the Analytica Decision Engine (ADE) from ASP.NET ===

'''Date and Time:''' Thursday, April 10, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Fred Brunton, Lumina Decision Systems

The Analytica Decision Engine (ADE) allows you to utilize a model developed in Analytica as a computational back-end engine from a custom application. In this webinar, we'll create a simple active web server application using ASP.NET that sends inputs submitted by a user to ADE, and displays results computed by ADE on a custom web page. In doing this, you will get a flavor how ADE works and how you program with it. If you've never created an active server page, you may enjoy seeing how that is done as well. This introductory session is oriented more towards people who do not have experience using ADE, so that you can learn a bit more about what ADE is and where it is appropriate by way of example.

You can watch a recording of this webinar at [http://AnalyticaOnline.com/WebinarArchive/2008-04-10-ADE-from-ASPNET.wmv ASP-from-ASPNET.wmv]. To download the program files that were created during this webinar [[Media:WebSite6.rar| Click here]].

= Optimization =

=== [[Analytica_4.3#Structured_Optimization|Introduction to Structured Optimization]] ===

'''Date and Time:''' Thursday, Febrary 24, 2011 at 10:00am PST (1:00pm EST, 6:00pm GMT)

'''Presenter''': Paul Sanford, Lumina Decision Systems

'''Abstract'''

[[Analytica 4.3]] is now available for beta testing and will be released in early March. The new version includes expanded optimization capabilities and simplified workflow for encoding optimization problems. The new '''Structured Optimization''' framework in 4.3 is centered around a new function, [[DefineOptimization]](), which replaces all three of the previous type-specific functions: LPDefine(), QPDefine() and NLPDefine(). It also introduces a new node type, '''Constraint''', which allows you to specify constraints using ordinary expressions. Paul will build up some basic examples using Structured Optimization and field questions from users.

A recording of this webinar can be viewed at: [http://AnalyticaOnline.com/WebinarArchive/2011-2-24-Structured-Optimization.wmv Structured-Optimization.wmv]. The example models used during this webinar are: [[File:Beer Distribution LP1]],[[File:Beer Distribution LP2]]

=== Introduction to Linear and Quadratic Programming ===

Date and Time: Thursday, Oct 11, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

This talk is an introduction to linear programming and quadratic programming, and an introduction to solving LPs and QPs from inside an Analytica model (via Analytica Optimizer). LPs and QPs can be efficiently encoded using the Analytica Optimizer functions [[LpDefine]] and [[QpDefine]]. I'll introduce what a linear program is for the sake of those who are not already familiar, and examine some example problems that fit into this formalism. We'll encode a few in Analytica and compute optimal solutions. Although LPs and QPs are special cases of non-linear programs (NLPs), they are much more efficient and reliable to solve, avoid many of the complications present in non-linear optimization, and fully array abstract. Many problems that initially appear to be non-linear can often be reformulated as an LP or QP. We'll also see how to compute secondary solutions such as dual values (slack variables and reduced prices) and coefficient sensitivies. Finally, [[LpFindIIS]] can be useful for debugging an LP to isolate why there are no feasible solutions.

You can watch a recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-10-11-LP-QP-Optimization.wmv LP-QP-Optimization.wmv] (requires Windows Media Player)

The model file created during this webinar is here: [[media:LP QP User Group.ANA|LP QP Optimization.ana]]

=== Non-Linear Optimization ===

Date and Time: Thursday, Oct 4, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Lonnie Chrisman, Lumina Decision Systems

Abstract

This talk focuses on the problem of maximizing or minimizing an objective criteria in the presence of contraints. This problem is referred to as a non-linear program, and the capability to solve problems of this form is provided by the Analytica Optimizer via the [[NlpDefine]] function. In this talk, I'll introduce the use of [[NlpDefine]] for those who have not previously used this function, and demonstrate how NLPs are structured within Analytica models. I'll examine various challenges inherent in non-linear optimization, tricks for diagnosing these and some ways to address these. We'll also examine various ways in which to structure models for parametric analyses (e.g., array abstraction over optimization problems), and optimizations in the presence of uncertainty.

You can watch a recording of this session here: [http://AnalyticaOnline.com/WebinarArchive/2007-10-04-Nonlinear-Optimization.wmv Nonlinear-Optimization.wmv]

During the talk, these two models were created:
* [[media:Simple Nonlinear optimization.ANA|Simple Nonlinear Optimization.ana]]
* [[media:Nonlinear asset allocation.ANA|Nonlinear asset allocation.ana]]

= Vertical Applications and Case Studies =

=== Regional Weather Data Analysis ===

'''Date and Time:''' Thursday, 22 April 2010 10:00am Pacific Daylight Time

'''Presenter:''' Brian Parsonnet, Ice Energy

'''Abstract'''

There are numerous sources of weather data on the web. Users of this data face a few common problems: how to gather the data in volume, how to normalize the data regardless of source, and how to analyze the results to generate insight. Analytica is the perfect tool to address all three issues simply and efficiently. A sample model will be shown illustrating some of techniques.

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2010-04-22-Regional-Weather-Analysis.wmv Regional-Weather-Analysis.wmv]. The model shown during the talk is [[media:Weather analysis.ana|Weather analysis.ana]], and the data file used by this model for Burbank weather can be downloaded from [[media:Burbank.zip|Burbank.zip]] (remember to Unzip it first to Burbank.txt). To avoid issues with ownership of the data, the temperatures in this file have been randomized (so the data is not accurate) and other fields zeroed out, but this will still allow you to play with the model and data.

=== Automated Monitoring and Failure Detection ===

'''Date and Time:''' 5 Feb 2009, 10:00am Pacific Standard Time

'''Presenter:''' Brian Parsonnet, ICE Energy

'''Abstract'''

In many complex physical systems, the automatic and proactive detection of system failures can be highly beneficial. Often dozens of sensor readings are collected over time, and a computer analyzes these to detect when system behavior is deviating from normal. Sounding an alert can then facilitate early intervention, perhaps catching a component that is just starting to go bad.

In a complex physical system with multiple operating modes and placed in a changing environment, anomaly detection is a very difficult problem. Simple sensor thresholds (and other related approaches) lack context-dependence, often making these simple approaches insufficient for the task. What is normal for any given sensor depends on the system's operating mode, time of day, activities in progress, and environmental factors. Simple thresholds that don't take such context into account either end up being so loose that they miss legitimate anomalies, or so tight that too many excess alarms are generated during normal conditions.

In this webinar, I'll show an expert system I've developed in Analytica that detects anomalies and developing failures in our deployed cooling system products. Data from dozens of sensors is collected in 5 minute intervals and the system transitions through multiple operating modes, daily and seasonal environmental fluctuations, and system demands. The Analytica model provides a framework in which complex rules that take multiple factors into account can be expressed, and used to estimate acceptable upper and lower operating ranges that are dynamically adjusted across each moment in time, taking into account whatever context is available. The Analytica environment presents a very readable and understandable language for expressing monitoring rules, and the overall transparency enables us to spot where other rules are needed and what they need to be.

[[Image:AutoMonitoring.png|frame|none|Graph illustrates how upper and lower bounds on operating range is adjusted to context. Actual sensor data is green, the red and blue lines show the computed bounds on acceptable operating range at each point in time.]]

A recording of this webinar can be watched at [http://AnalyticaOnline.com/WebinarArchive/2009-02-05-Failure-detection.wmv Failure-Detection.wmv].

=== Data Center Capacity Planning ===

''Please note that this presentation will be on Wednesday rather than Thursday this week.''

'''Date and Time:''' Wednesday, October 21, 2008 10:00am Pacific Daylight Time

'''Presenter:''' Max Henrion, Lumina Decision Systems

'''Abstract'''

Data center energy demands are on the rise, creating serious financial as well as infrastructural challenges for data center operators. In 2006, data centers were responsible for a costly 1.5 percent of total U.S. electricity consumption, and national energy consumption by data centers is expected to nearly double by 2011. For data center operators, this means that many data centers are reaching the limits of power capacity for which they were originally designed. In fact, Gartner predicts that 50 percent of data centers will discover they have insufficient power and cooling capacity in 2008.

This week's presentation will provide an overview of [http://www.lumina.com/ADCAPT/Analytica%20Data%20Center%20Capacity%20Planning%20Tool%20Overview.pdf ADCAPT -- the Analytica Data Center Capacity Planning Tool]. For this webinar, the User Group will be joining a presentation that is also being given outside of the Analytica User Group, but I (Lonnie) think is also of interest to the User Group community in that it shows of an example of a re-usable Analytica model, containing several very interesting and novel techniques, applied to a very interesting application area.

Due to technical difficulties, this webinar was not recorded.

=== Modeling the Precision Strike Process ===

'''Date and Time:''' Thursday, October 16, 2008, 10:00am Pacific Daylight Time

'''Presenter:''' Henry Neimeier, MITRE

'''Abstract'''

We describe a new paradigm for modeling, and apply it to a simple view of the precision strike attack process against mobile targets. The new modeling paradigm employs analytic approximation techniques that allow rapid model development and execution. These also provide a simple dynamic analytic risk evaluation capability for the first time. The beta distribution is used to summarize a broad range of target dwell and execution time scenarios in compact form. The data processing and command and control processes are modeled as analytic queues.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2008-10-16-Precision-Strike-Process.wmv Precision-Strike-Process.wmv]. Several related papers and materials are also available, including:
* [[media:MILCOM96.pdf|A New Paradigm For Modeling The Precision Strike Process (U)]] by H. Neimeier from MILCOM96
* [[media:Milcom96.ana|Milcom96.ana]] -- the model from the talk and above paper.
* [[media:PhalanxMar.pdf|Analytic Uncertainty Modeling Versus Discrete Event Simulation]] by H. Neimeier, PHALANX March 1996.
* [[media:AnalQueNet.pdf|Analytica Queuing Networks]] by H. Neimeier, Proc. 12th Int'l Conf. Systems Dynamics Soc. 1994.
* [[media:Kuskey CAPE MTR-11.pdf|The Architecture of CAPE Models]] by K.P. Kuskey and S.K. Parker, MITRE Tech. Report.
* [[media:Cmac2pap_2_.pdf|Analytical Modeling in Support of C4ISR Mission Assessment (CMA)]] by F.R. Richards, H.A. Neimeier, W.L. Hamm, and D.L. Alexander, 3rd Int'l Symp. on Command and Control Research and Technology, 1997.
* [[media:INCOSE96.pdf|Analyzing Processes with HANQ]] by H. Neimeier and C. McGowan, MITRE, from INCOSE96.
* Functions for drawing ''radar plots'': [[media:Radarplt.ANA|Radarplt.ana]]
* Power point slides: [[media:Cape.ppt|Cape.ppt]], [[media:PGMrisk.ppt|PGMrisk.ppt]], and [[media:JDEMweb.ppt|JDEMweb.ppt]]

=== Modeling Utility Tariffs in Analytica ===

Presenter: Brian Parsonnet, Ice Energy

Date and Time: Thursday, Nov 8, 2007 at 10:00 - 11:00am Pacific Standard Time

Modeling utility tariffs is a tedious and complicated task. There is no standard approach to how a utility tariff is constructed, and there are 1000’s of tariffs in the U.S. alone. Ice Energy has made numerous passes at finding a “simple” approach to enable tariff vs. product analysis, including writing VB applications, involved Excel spreadsheets, using 3rd party tools, or outsourcing projects to consultants. The difficulty stems from the fact that there is little common structure to tariffs, and efforts to standardize on what structure does exist is confounded by an endless list of exceptions. But using the relatively simple features of Analytica we have created a truly generic model that allows a tariff to be defined and integrated in just a few minutes. The technique is not fancy by Analytica standards, so this in essence demonstrates how Analytic’s novel modeling concept can tackle tough problems.

You can watch a recording of this webinar at: [http://AnalyticaOnline.com/WebinarArchive/2007-11-08-Tariff-Modeling.wmv 2007-11-08-Tariff-Modeling] (Requires Windows Media Player)

=== Modeling Energy Efficiency in Large Data Centers ===

Date and TimeThursday, Oct 25, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter:Surya Swamy, Lumina Decision Systems

Abstract

The U.S. data center industry is witnessing a tremendous growth period stimulated by increasing demand for data processing and storage. This has resulted in a number of important implications including increased energy costs for business and government, increased emissions from electricity generation, increased strain on the power grid and rising capital costs for data center capacity expansion. In this webinar, Analytica's dynamic modeling capabilities coupled with it's advanced uncertainty capabilities, which offer tremendous support in building cost models for planning and development of energy efficient data centers, will be illustrated. The model enables users to explore future technologies, the performance, costs and efficiencies of which are uncertain and hence to be probabilistically evaluated over time.

You can watch a recording of this presentation at: [http://AnalyticaOnline.com/WebinarArchive/2007-10-25-Data-Center-Model.wmv Data-Center-Model.wmv] (Requires Windows Media Player)

= Graphing =

=== Creating Scatter Plots ===

'''Date and Time:''' Thursday, May 15, 2008 at 10:00 - 11:00am Pacific Daylight
Time

'''Date and Time:''' Thursday, Aug 23, 2007 at 10:00 - 11:00am Pacific Daylight
Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

This webinar focuses on utilizing graphing functionality new to Analytica 4.0, and specifically, functionality enabling the creative use of scatter plots. The talk will focus primarily on techniques for simultaneously displaying many quantities on a single 2-D graph. I'll discuss several methods in which multiple data sources (i.e., variable results) can be brought together for display in a single graph, including the use of result comparison, comparison indexes, and external variables. I'll describe the basic new graphing-role / filler-dimension structure for advanced graphing in Analytica 4.0, enabling multiple dimensions to be displayed on the horizontal and vertical axes, or as symbol shape, color, or symbol size, and how all these can be rapidly pivoted to quickly explore the underlying data. I'll discuss how graph settings adapt to changes in pivot or result view (such as Mean, Pdf, Sample views).

A recording of this webinar can be viewed at [http://AnalyticaOnline.com/WebinarArchive/2008-05-15-Scatter-Plots.wmv Scatter-Plots.wmv].

Model used: During this webinar, I started with some example data in the model [[media:Chemical elements orig.ANA|Chemical elements.ana]]. The original file is in the form before graph settings were changed. By the end of the webinar, many graph settings had been altered, and various changes made, resulting in [[media:UG_Webinar_Scatter_Plots.ana|Scatter-Plots.ana]] (during the Aug 23 presentation, this was the final model: [[media:Chemical elements2.ANA|Chemical elements2.ana]]).

=== Graph Style Templates ===

''' Date and Time:''' Thursday, February 28, 2008, 10:00 - 11:00 Pacific Standard Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

Graph style templates provide a convenient and versitile way to bundle graph setup options so that they can be reused when viewing other result graphs. For example, if you've discovered a set of colors and fonts and a layout that creates the perfect pizzazz for your results, you can bundle that into a template where you can quickly select it for any graph. In this talk, I'll introduce how templates can be used and how you can create and re-use your own. I'll show the basics of using existing templates, previewing what templates will look like, and applying a given template to a single result or to your entire model. We'll also see how to create your own templates, and in the process I'll discuss what settings can be controlled from within a template. I'll discuss how graph setup options are a combination of global settings, template settings, and graph-specific overrides. I'll show how to place templates into libraries (thus allowing you to have template libraries that can be readily re-used in different models), and even show how to control a few settings using templates that aren't selectable from the Graph Setup UI. I'll also touch on how different graph setting are associated with different aspects of a graph, ultimately determining how the graph adapts to changes in uncertainty view or pivots.

The model created during this webinar is here: [[media:Graph style templates.ana|Graph style templates.ana]].
You can watch a recording of the webinar here: [http://AnalyticaOnline.com/WebinarArchive/2008-02-28-Graph-Style-Templates.wmv Graph-Style-Templates.wmv].

= Scripting =

=== Button Scripting ===

Date and Time: Thursday, Sept. 6, 2007 at 10:00 - 11:00am Pacific Daylight Time

Presenter: Max Henrion, Lumina Decision Systems

Abstract

This webinar is an introduction to Analytica's typescript and button scripting. Unlike variable definitions, button scripts can have side-effects, and this can be useful in many circumstances. I'll cover the syntax of typescript (and button scripts), and how scripts can be used from buttons, picture nodes or choice inputs. I'll introduce some of the Analytica scripting language to those who may have seen or used it before. And we'll examine some ways in which button scripting can be used.

You can watch the recording of this webinar here: [http://AnalyticaOnline.com/WebinarArchive/2007-09-06-Button-Scripting.wmv Button Scripting.wmv] (Requires Windows Media Player or equiv). The model files and libraries used during the webinar are in [[media:Ana_tech_webinar_on_scripting.zip|Ana_tech_webinar_on_scripting.zip]].

= Analytica User Community =

=== The Analytica Wiki, and How to Contribute ===

'''Date and Time:''' (tentative) Thursday, October 30, 2008, Pacific Daylight Time

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The Analytica Wiki is a central repository of resources for active Analytica users. What's more, you -- as an active Analytica user -- can contribute to it. As an Analytica community, we have a lot to learn from each other, and the Analytica Wiki provides one very nice forum for doing so. You can contribute example models and libraries, hints and tricks, and descriptions of new techniques. You can fix errors in the Wiki documentation if you spot them, or add to the information that is there when you find subtleties that are not fully described. If you spend a lot of time debugging a problem, after solving it you could document the issue and how it was solved for your own benefit in the future, as well as for others in the user community who may encounter the same problem. When you publish a relevant paper, I hope you will add it to the page listing publications that utilize Analytica models.

I will provide a quick tour of the Analytica Wiki as it exists today. I'll then provide a tutorial on contributing to the Wiki -- e.g., the basics of how to edit or add content. The [http://Wikipedia.org Wikipedia] has had tremendous success with this community content contribution model, and I hope that after this introduction many of you will feel more comfortable contributing to the Wiki as you make use of it.

Due to a problem with the audio on the recording, the recording of this webinar is not available.

= Licensing or Installation =

=== Reprise License Manager Tutorial ===

'''Date and Time:''' Wednesday, 11 March 2010, 10:00am Pacific Standard Time

'''Presenter:''' Bob Mearns, Reprise Software Inc.

'''Abstract'''

The Reprise License Manager (RLM) allows all Analytica and ADE licenses within an organization to be managed from a central server. RLM can be used with either ''floating'' or ''named-user'' licenses.

This tutorial on RLM administration is being given by Bob Mearns, lead software developer at Reprise Software, Inc., who has over 15 years' experience developing and supporting software license managers. This session will focus on:

* Basic RLM Server Setup
* How and where RLM looks for licenses
* Using the RLM Web Server Admin Interface
* Using RLM diagnostics, new in RLM v8
* A systematic approach to diagnosing license server connectivity problems

There is a big focus in this talk on how to debug problems with the RLM license manager, and in the process many of the technical details pertaining to the RLM setup are covered. This talk is most relevant for IT managers who administer the license server, and for people who may be installing the RLM server who would like a more thorough understanding of how things work. The RLM license manager is used to host centrally managed licenses, which includes floating and named-user licenses.

This talk is being provided by Reprise Software.

This webinar may be viewed here: [http://analyticaonline.com/WebinarArchive/2010-03-11-RLM-troubleshooting.wmv RLM-troubleshooting.wmv]. The trouble-shooting tips document covered in the talk is at [http://reprisesoftware.com/RLM_Troubleshooting_Tips.pdf RLM Troubleshooting Tips].

See also:
* [[Configuring an RLM Server]] -- step-by-step for installing the RLM server
* [[How to Install Analytica 4.2 -- Centrally Managed License]] -- for the Analytica user's side installation

File:Beer Distribution LP1.ana

2011-02-25T00:48:28Z

Psanford: A classic LP example using structured optimization

A classic LP example using structured optimization

File:Beer Distribution LP2.ana

2011-02-25T00:46:54Z

Psanford: Beer Distribution with extrinsic index added. This model demonstrates Analytica's ability to abstract optimizations over extrinsic indexes, resulting in multiple optimization runs.

Beer Distribution with extrinsic index added. This model demonstrates Analytica's ability to abstract optimizations over extrinsic indexes, resulting in multiple optimization runs.

Analytica User Group

2011-02-21T23:13:52Z

Psanford: /* Analytica_4.3#Structured_Optimization */

[[/Past Topics|Past Topics]] >>

The Analytica User Group is a way for Analytica users to support each other by sharing tips and function libraries. It includes a webinar series with tutorials and demonstrations on key Analytica features.

= Webinar Series =

Analytica User Webinars are a great way to learn about key Analytica features and modeling skills. They are live demos, using Citrix Gotowebinar, and VOIP or your telephone, at your choice. Questions, comments, and tangents are welcome.

There are webinars most weeks, usually on Thursdays at 10am PST (1pm EST). Seats are limited. To sign up for a particular webinar, see "How to Attend" below. Presentations may last anywhere from 20 to 90 minutes (with an estimate provided upfront).

If you missed a User Webinar, don't despair. We usually records them, including audio and screensharing. So, you can replay them any time. See below for the full list.

;Recent talks include:
* [[/Past_Topics#Gentle_Introduction_to_Modeling_Uncertainty:_Webinar_Series|Gentle Introduction to Modeling Uncertainty]]
** [[/Past_Topics#Session_1:_Uncertainty_and_Probability|Session 1]]: (29 Apr 2010)[[/Past_Topics#Session_1:_Uncertainty_and_Probability|Introduction to Uncertainty, Probability]].
** [[/Past_Topics#Session_2:_Probability_Distributions|Session 2]]: (6 May 2010)[[/Past_Topics#Session_2:_Probability_Distributions|Probabiliity Distributions]]
** [[/Past_Topics#Session_3:_Monte_Carlo|Session 3]]: (13 May 2010)[[/Past_Topics#Session_3:_Monte_Carlo|Monte Carlo Simulation]]
** [[/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Session 4]]: (20 May 2010) [/Past_Topics#Session_4:_Measures_of_Risk_and_Utility|Measures of Risk and Utility]]
** [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Session 5]]: (3 June 2010) [[/Past_Topics#Session_5:_Risk_Analysis_for_Portfolios|Risk Analysis for Portfolios]]
** [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Session 6]]: (10 June 2010) [[/Past_Topics#Session_6:_Common_Parametric_Distributions|Common Parametric Distributions]]
** [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Session 7]]: (24 June 2010) [[/Past_Topics#Session_7:_Expert_Assessment_of_Uncertainty|Expert Assessment of Uncertainty]]
** [[/Past_Topics#Session_8:_Hypothesis_Testing|Session 8]]: (15 July 2010) [[/Past_Topics#Session_8:_Hypothesis Testing|Statistical Hypothesis Testing]]
* [[/Past_Topics#Regional_Weather_Data_Analysis|Regional Weather Data Analysis]] (22 Apr 2010)
* [[/Past_Topics#Step_Interpolation|Step Interpolation]] (8 Apr 2010)
* [[/Past Topics#Spearman_Rank_Correlation|Introduction to Spearman's Rank Correlation]] (25 Mar 2010)

== Schedule of Upcoming Webinars ==

=== [[Analytica_4.3#Structured_Optimization|Introduction to Structured Optimization]] ===
'''Date and Time:''' Thursday, Febrary 24, 2011 at 10:00am PST (1:00pm EST, 6:00pm GMT)

'''Presenter''': Paul Sanford

'''Abstract'''
Analytica 4.3 is now available for beta testing and will be released in early March. The new version includes expanded optimization capabilities and simplified workflow for encoding optimization problems. The new '''Structured Optimization''' framework in 4.3 is centered around a new function, DefineOptimization(), which replaces all three of the previous type-specific functions: LPDefine(), QPDefine() and NLPDefine(). It also introduces a new node type, '''Constraint''', which allows you to specify constraints using ordinary expressions. Paul will build up some basic examples using Structured Optimization and field questions from users.

=== [[Model building by mouse]] ===

'''Date and Time:''' (TBD)

'''Presenter''': (TBD)

'''Abstract'''

See Model building by mouse for the concept. The talk will also cover the TemplateInput and TemplateOutput attributes first introduced in 4.2 that make template modules possible, and thus expand the potential power and applicability of this modeling technique.

=== Saving Memory: Controlling when results are (and are not) cached ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

When a variable is computed, Analytica caches the result (i.e., keeps a copy around) so that when the same result is needed later, it does not need to be recomputed. Large array-valued results can end up consuming large amounts of memory, and lead to annoyances such as running out of available memory. There are instances where it isn't really necessary to cache a result -- either it will never be needed a second time, or it isn't a big deal to recompute it, or it is an exact copy of its parent (perhaps because only the result graph settings are different), so that by configuring that variable to not cache its result, or to release its cache as soon as all its children are computed, memory space can be recovered, perhaps allowing larger simulations to be conducted.

Analytica 4.2 introduces a new feature allowing you to configure when results are (or are not) cached. I'll demonstrate how to utilize this feature, and discuss limitations, such as situations where it would be a very bad idea not to cache.

If time allows (it probably will), I'll also throw in two other small memory-related controls new in 4.2. The [[WhatIf]]-style functions ([[WhatIf]], [[WhatIfAll]], [[Dydx]], [[Elasticity]], [[NlpDefine]]), i.e., functions that compute hypotheticals, have been modified in 4.2 to preserve previously computed values of other variables. This is a nice feature, but can impact tight-memory cases. I'll explain this change and how to avoid the extra memory consumption where appropriate. Second, I'll also show how Analytica's maximum working set size can be configured (on some operating systems), which can be used to keep other applications on your computer highly responsive even when large memory-intensive computations are churning away.

In Analytica 64-bit, you are less likely to encounter the problem of running out of memory, but you still may encounter performance slowdown from huge memory utilization. Some large models, requiring extensive memory resources, run smoothly even when requiring far more memory than there is RAM available, while other result in extensive ''thrashing'', taking very long to evaluate. I've found that differentiating factor has to do with the size of individual arrays within the model. Thrashing tends to occur when a single array consumes more than about 1/3 the available RAM, while if all arrays in the model remain small relative to available RAM, huge memory evaluations often run very smoothly. I'll relate these observations, and what that means with respect to sample size.

This talk is appropriate for advanced Analytica modelers.

=== The [[Dynamic]] Function and Dynamic Loops ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

The [[Dynamic]] function in Analytica 4.2 is a bit more flexible than previously. It is now possible for [[User-Defined Functions]] to be part of a dynamic loop, and there are cases where it is now possible to operate over the [[Time]] index where previous it was not. I'll discuss how dynamic models are evaluated, and how that differs from non-dynamic models, and cover some tricks for tracing evaluation and debugging complex dynamic models (which can often be challenging to debug). I'll also introduce how [[Dynamic]] can be used on an index other than the built-in [[Time]] index.

=== [[IntraTable]]s ===

'''Date and Time:''' (TBD)

'''Presenter:''' Lonnie Chrisman, Lumina Decision Systems

'''Abstract'''

An [[IntraTable]] is a variation of an edit [[Table]], where (like [[Table]]s), the cells may contain expressions, but unlike [[Table]]s, the expressions in each cell can refer to values of other cells. This may (quite appropriately) conjure up nightmares of spreadsheet hell. While this isn't something you'd want to use nonchalantly, there are cases when the ability to reference other cells within the same table comes with its benefits. Because the [[IntraTable]] allows any pattern of references (it doesn't have to be strictly left-to-right, for example), as long as a cell-cycle isn't created, it is possible to encode certain recurrences that would otherwise be quite difficult to encode.

== How to Attend ==

To attend, you need to sign up by contacting Lumina at mailto:webinars@lumina.com. Please sign up at least a day prior. Attendance is limited, so please don't sign up unless you sincerely intend to attend.

These Webinars are FREE to users who have an up-to-date Support for Analytica. If you are unsure, check with mailto:sales@lumina. For those without current support, the fee is US$50.

== How to be a Presenter ==

Being a presenter at an Analytica webinar provides an opportunity to make others in the Analytica community aware of your successes or capabilities. Consultants may find this an opportunity for exposure to others with particular modeling needs. Also, if you are an Analytica aficionado, this is a great opportunity to help others.

If you would like to be a presenter, submit your proposed topic to webinars@lumina.com and possible presentation times (include the time zone). We will schedule the GotoMeeting conference (you do not need a gotoMeeting subscription yourself) and we will make you presenter during the session, allowing you to share your screen while you talk. You will most likely make use of Power Point and a running Analytica during your presentation.

==Analytica User Group:([[Analytica_User_Group/Past_Topics#The_Large_Sample_Library|Large Sample Library]])==

== [[Analytica User Group/Past Topics|Archive of Past Topics]] ==

Visit the [[Analytica User Group/Past Topics|Archive of Past Topics]] for abstracts, example models, and recordings of previous Analytica User Group webinars. Previous topics that have been presented include:

* Guidelines for Model Transparency
* Automated Monitoring and Failure Detection
* Principal Component Analysis (PCA)
* Internal Rate of Return ([[IRR]]) and Modified Internal Rate of Return ([[MIRR]])
* Bond Portfolio Analysis
* Net Present Value ([[NPV]])
* The Analytica Wiki, and How to Contribute
* Data Center Capacity Planning
* Modeling the Precision Strike Process
* The Performance Profiler
* Variable Stiffness Cubic Splines
* Introduction to [[DetermTable]]s
* Importance Sampling (Bayesian posteriors)
* Importance Sampling (rare events)
* Statistical Functions
* [[Table Splicing]]
* Analytica Web Player
* [[SubTable|SubTables]]
* Creating Custom Distribution Functions
* Using the Check attribute to validate inputs and results
* The [[Dynamic]] Function
* [[Logistic Regression]]
* Creating Control Panels
* Statistical Functions
* Creating Scatter Plots
* Using [[Regression]]
* New Functions for Reading Directly from an Excel File
* Advanced Tornado Charts -- when inputs are Array-Valued
* Using the Analytica Decision Engine (ADE) from ASP.NET
* OLE Linking
* Tornado Charts
* Correlated and Multivariate Distributions
* Assessment of Probability Distributions
* Graph Style Templates
* Sneak preview of Analytica Web Publisher
* Querying an OLAP server
* Querying an ODBC relational database
* [[Self-Indexed Arrays|Self-Indexes]], Lists and [[Implicit Dimensions]]
* Flattening and Unflattening of Arrays
* Introduction to Arrays and Array Abstraction
* [[Local Indexes]]
* The [[Iterate]] Function
* The [[Using_References|Reference and Dereference Operators]]
* Modeling Utility Tariffs in Analytica
* Modeling Energy Efficiency in Large Data Centers
* Calling External Applications
* Introduction to Linear and Quadratic Programming
* Non-Linear Optimization
* Writing [[User-Defined Functions]]
* Modeling Markov Processes in Analytica
* Manipulating Dates in Analytica
* Button Scripting
* Manipulating Indexes and Arrays in Analytica Expressions
* Edit Table Enhancements in Analytica 4.0
* [[Handle]]s and [[Meta-Inference]]

== Potential future topics ==

If you would like to see a webinar on a given topic, please feel free to add it here. If you see a topic listed and would like to be a presenter, let us know.

* Statistical hypothesis testing.
** Standard textbook tests
** Computing p-values using Monte Carlo for complex and non-standard statistical models.

* Time-series analysis

* Using the interpolation functions, [[LinearInterp]], [[CubicInterp]], [[StepInterp]].

* Numeric precision - numeric round-off, underflow, etc., why they happen, what to do about it.

* Net Present Value and Internal Rate of Return -- introduction to the use of these commonly used metrics for quantifying decision quality.

* Using the ''Performance Profiler'' to understand where your model consumes time and memory resources.

* Producing graphs from ADE: Including how to serve graphs from web pages.

* Intracacies of the Domain attribute

* Mastering Array Abstraction

* Writing User-Defined Distribution functions.

* Getting data into Analytica.

* [[DetermTable]]s

* Large-scale sampling. Techniques when memory limitations constrain sampleSize.

* Creating User-Defined functions. Review of [[Function Parameter Qualifiers]].

* Understanding [[Evaluation Contexts]]

* Bayesian Inference

* Sensitivity Analysis.

* Dynamic Programming

* Approximate and Stochastic Dynamic Programming

== User Survey Results ==

During the first week of September, we sent out a survey to people who had attended webinars so far. Please continue providing us with feedback. Here is some feedback to date:

Level of difficult and speed:

So far, of those who answered this question, 25% say "a bit too easy", 25% say "too hard/fast", and 50% say "just right".

Topics requested for future webinars:

* How to run multiple iterations, e.g. 100 iterations with uncertainty sample of 1000.

* Sampling for rare events.

* Using the lognormal function.

* Financial modeling

* New array functionality (subtables, choices in tables)

* Optimizer !!!

* Dynamic models (twice requested)

* Choice of distributions

* Re-sampling and radomize methods and uncertainty sample size

* Tricks for sensitivity analysis

* Input and output nodes

* Importance analysis

= The Analytica Wiki =

The [[Analytica Wiki ]] contains many resources, including in-depth reference materials, relevant articles, example models, tutorials, etc., to help users master Analytica and find what they need. Even better, Analytica users can contribute! You can upload your own models, articles, expand on or correct materials that are there, etc., for the benefit of the entire Analytica community.