What's new in Analytica 5.0?

Here are some highlights of new features in Analytica 5.0, released on October 30, 2017. See below for a video intro and more details.

New icons and pop-up node controls update the look and give easier and faster access to common actions on each node:

A new introduction screen appears when you start Analytica giving quick access to recent models, example models, and to search the docs
This Analytica documentation uses a clearer design, better search, now with easy access and search from inside Analytica. It includes the Tutorial, User, Optimizer and Developer Guides previously distributed as PDF documents.
Tables now offer cell-level formatting with number formats, borders, fonts, alignment, colors, and images by cell, row, column, slice, or rectangular area. Computed cell formats let you change colors and other formats to highlight key results.
A more powerful Find function exists, with icon in toolbar, to search for a variable, a function, or any text in the model and docs.
Multi-thread computing speeds up Analytica using multiple processors or cores for parallel computation.
New user input and output options
Sobol sampling, a quasi-Monte Carlo simulation method that often lets you generate probability distributions with greater accuracy with many fewer samples, and the new Keelin Metalog distribution and convenient UncertainLMH distribution.
New functions make it easy to import and export CSV and JSON file formats, widely used for data exchange.

Video: What's new in 5.0 tour video (31 minutes)

This Analytica documentation

If you have been using this Analytica docs over the last year, you will have noticed major improvements:

It has a cleaner and simpler design, with key menus and links across the top.
The search box is much smarter and faster. It shows partial matches as you type.
The manuals previously distributed as PDFs -- Analytica Tutorial, Analytica User Guide, Analytica Optimizer Guide, and ADE User Guide -- are now online in this Analytica docs. (Analytica 5.0 and later no longer includes PDF manuals.) So now you can search all manuals and other docs at once using the search box, without having to worry about which document to look in.
It's much easier to find what you need with linked contents on the new Analytica Docs home page and index pages, especially Functions by category, Analytica User Guide, Contents of User Guide.
We've added lots of cross-references between related pages, especially in the See also section at the bottom of each page.
When you select Edit from This page menu at the top of the page, you get the WYSIWYG (What You See Is What You Get) visual editor. It lets you edit pages, styles, headings, insert links, and more without needing to learn the wiki markup language. (If you like the wiki markup, or the Visual editor can't do what you need, you can still select Edit Source from This page menu.)
As always, let us know if there's still stuff you want and can't find.
And it's a wiki, which means you can improve and add stuff yourself!

Quick links to the Wiki

Analytica has lots of links that open this wiki in your web browser to find what you needi:

In the Help menu, select User Guide, Tutorial, or Optimizer to see each document's contents page in the wiki (in previous releases, they opened a PDF document):
Each dialog window has a ? icon on its title bar. Click it to open the wiki page explaining that dialog:

Open the Find dialog (ctrl+F or click the Find icon in the Toolbar), and click "on the wiki" to search it for the text you entered:
Here are ways to open the page explaining a built-in function or system variable:
- Select a function name (double-click it) in a Definition, open the right-click menu, and select Wiki help on «function».
- Use the Find dialog to search for the function, and click "Find in wiki" button
- Open a function's object window from the Definition menu and click the blue "Wiki help" link.
- Right-click the function name in the Outline window and select Wiki help

Finding the function you need

Here are some ways to find a function when you don't know its name:

Open the Definition menu
- Browse the menu hierarchy with more meaningful groups of functions (and operators and system variables).
- Move the mouse over each function to see a tooltip explaining what it does.
- Select a function to open an Object window showing its parameters and description.
- In the Object window for a system function, click the blue "Wiki help on <function>" to open its wiki page with more details.
In the Outline window open the new System Libraries:

- Open sub-libraries to explore categories of function,
- select a function to see its description below,
- open the right-click menu on a function and select Go to object window or wiki help.
- Double-click on Operators library to see a cheat sheet on the operators +, - , <>, IF THEN ELSE, @, \ # etc.
Function index option on Help menu takes you to wiki Function index.

Find Dialog

The Find Dialog is much expanded, now accessible from the Find icon in the Toolbar as well as ctrl+F. It lets you:

Select which attribute(s) to search -- Identifier, Title, any, or all text attributes.
Search the Wiki directly
Select a tab to control if you want to search for objects in the model or a selected Table.
Use a regular expression to define wild-card and more complex searches
Mark matching nodes with in Diagrams and Outline view.
Show a list of matching objects as the Found set in the Outline view.
The System Variable Sys_FoundSet let's you access results in an expression.

FindObjects function

The new FindObjects function performs powerful searches in an expression. You can search for text by attribute, and specify which objects to search by class, module, dependencies, inputs or outputs of a variable, and more. It returns the found objects as a set. You can use standard set functions for compound search criteria.
You can assign a resulting set to Sys_FoundSet, for example in an OnClick expression, to show the results to an end user. For example, you could Find all variables in the same dynamic loop as X.

Influence diagrams and user interface

New icons and pop-up node controls

The most obvious changes in 5.0 are the new icons in the toolbar and the pop-up controls that appear when you move your mouse cursor over a node in a diagram.

We've added a Find icon button to open the dramatically expanded Find Dialog
Here's a short video walking through the toolbar update.
The Module hierarchy bar now shows by default across the top of each Diagram window to give you context when your model has nested modules. You can click on any module listed there to quickly jump up to a parent or higher ancestor.
When you move your cursor over a node, "Pop-up node controls" appear around it, using the new icons. Click these controls to open an Object, Result, or Expression view. This is much easier and faster than moving the mouse up to the toolbar -- especially with a large screen.
Press the arrowheads on the left and right of each node to see a list of its Inputs or Outputs. Select an input or output from the list to jump to that node in its Diagram. These arrowheads make this useful way to navigate a model much clearer -- which was previously hard-to-see and little-known.
Drag from the arrow pop-up node control (in edit mode) to draw an influence arrow to another node. This replaces the separate arrow-drawing mode in the toolbar. You no longer have to go up to the Toolbar and change modes to draw an arrow. Here's a short video on how to draw an influence diagram in 5.0.

User inputs and outputs

The new MultiChoice user input control lets users select multiple options, where Choice lets users select only one (or All).
Set a user input node to accept only a literal number, a text value (without quotes), or any expression.

For User Input nodes, position the checkbox, choice menu, field, or other control button to the left of the label instead of the default right, from the Node Style Dialog:

Resize an image after pasting it into a node.
New options for the Calc, Result, Table (and other) buttons in User input and output nodes: By default, they are now non-bold. You can set them to icons, blank, or back to the current default bold from the Diagram Style dialog.

Choice controls now appear with options in normal (not bold) text.
Configure a textbox input node to show cue text, as is in the "Search the Analytica Docs" text box on the intro screen. The EditBoxCueText attribute specifies the cue text to show.

Node styles

Set node text to be left or right justified, in addition to the default centered in the Node Style Dialog.

Modify the text color of node titles using a new button on the Color palette:

Hide or show nodes using the NodeInfo attribute and disable (gray out) User Input nodes. Use the new ChangeNodeVisibility function from OnClick or OnChange event expressions to dynamically show or hide, enable or disable nodes. This clever blackjack tutorial video is a great showcase of changing node visibility.
Create a custom tabbing order between the controls in your user interface, which controls which control a user advances to next after changing a value, or when pressing the TAB key.

Text nodes with Title

Text nodes can now have both a title (in bold) and a description in the same node.

Text nodes have pop-up node controls that let you to edit Title and Description text, promote Description to Title, or demote Title to Description.
This feature is available only when you have Clear Type fonts enabled, which will be true for any model created in Analytica 4.4 or later. If you are using an old legacy model, enable Clear Type fonts from the Diagram Style dialog from your top-most model diagram. See Text nodes for details.

Graphics and images

After pasting an image into a diagram into a node, you can resize it by dragging its corner boxes. Here's a tutorial video on how this is done.
A new set of Canvas functions let you write expressions to generate a graphic image, which you can show on a diagram, in a node, or in a table. This tutorial video shows you how to crop an image with an expression.
You can also paste an image from another application directly into a cell of an edit table.

Formats and Tables

Cell-level formatting

You can now set number and cell formats for an entire table, or for a selected cell, row, column, header, slice, or range of cells in a table. Formats include:

Alignment: Horizontal, vertical, and indention.
Fonts: Face, size, bold, italics, underline, color
Borders: Side, thickness, line style, color
Colors: Background, font, borders.
Entry mode: Specify whether an end user may enter a only a number, text, a literal, or any expression into an Edit table.

To set a cell formats, select a cell, column, row, range, or entire table (ctrl+A) you want to change, and then select Number Format... or press "Ctrl+B). You can then select the number and cell formats from the Format dialog.

Any formats automatically adjust with changes in table pivot, indexes, splicing, etc.

Images in tables

You can copy an image from any application and paste it into a table cell:

Images will resize to fit the cell when you drag the cell borders.
You can obtain images using ReadImageFile or ReadFromURL functions.

Number formats

There are three new options in the Number format dialog:

General: It normally shows numbers in decimal format, but reverts to scientific (exponential) notation when needed for extreme magnitudes or to show the specified number of significant digits.
Hexadecimal: Denotes integer values in base-16 (digits 0-9 and a-f).
Binary: Denotes integer values is base-2 (0s and 1s)

Computed cell formats

You can compute the formats used by a result table using a Cell Format Expression for powerful "conditional formatting" -- for example:

Highlight negative or extreme values with a color or bold font. Here's a video tutorial on how.
Make a heat-map using the alpha channel of CellFill to map the result value of each cell as a color.
Identify groups of related cells using the CellBorder "Outer" option for «side».
Use CellBar under the text to show magnitude as an underline.
Use CellOnClick to install a custom action, such as a call to OpenURL for external hyperlinks or ShowWindow for internal (within-model) hyperlinks.

Please see Computed cell formats for details and examples.

Column widths and row heights

You can now manually change the row height of an individual row in a table.
Manual changes to column widths and row heights are now saved, so they persist when you close and reopen table, or reload the model.
The new auto-adjust column width and auto-adjust row height options in the context menus for column and row headers let you restore the default auto-adjusting behavior after you have manually fixed the column width or row height . You can also activate these options by double-clicking the right side column boundary in the column header, or bottom row boundary in the row headers.

SubTables and MultiTables

You can now use If-Then-Else in a SubTable or MultiTable meta expression, where the Then and Else expressions identify a source table or computed result.

Reset to cell default

A right-click menu option, Reset selected to default, in edit tables allows you to reset the contents of the selected to the table cell default for the table. The table cell default is controlled by the Cell Default attribute, which you can customize by turning on the Cell Default attribute in the Attributes dialog. This new option is particularly useful when you change the table cell default after table cells have already been populated with the previous default.

Unquoting cells

A right-click menu option in edit tables and index list edits lets you remove quotes from cells when an extra level of quotes exist -- for example, after a copy/paste adds an extra quotes that you don't want.

Introduction screen

When you start-up Analytica by double clicking the Analytica file icon, it greets you with a new introduction screen:

It includes

Large icons to Start a new model, Open an existing model, the Analytica Tutorial, and selected selected example models. Saves you initial clicks to bring up menus.
Quick access to the Recent file list.
A search box to search the Analytica docs for a function or phrase.

Graphing

Sort by data spread

A new graph style option, Sort by data spread, makes it easier to create Tornado plots. When you check this checkbox, the categorical labels are automatically sorted from greatest data spread to least, so your Tornados come out in order, without having to sort the data yourself in the Definitions. Even better, when there are extra dimensions, the sort order adjusts as you change the values of the slicers.
By unchecking the new graph chart style option, Enable bar outlines, you can remove the fine black outline around bars in a bar chart for a cleaner appearance.

Uncertainty

Sample size

Samplesize now starts out in a model with a default of 1000 instead of 100 to reflect today's larger and faster computers. You can still make it larger or smaller in the Uncertainty Setup dialog.

Keelin Metalog distribution

The new UncertainLMH( xLow, xMedian, xHigh, pLow, lb, ub) distribution gives an convenient way express uncertainty, by fitting a smooth continuous distribution to the 10th, 50th, and 90th percentiles (other percentiles are possible), and optional lower and upper bounds. It is based on the Keelin or "metalog" distribution, recently published by Tom Keelin, which fits a smooth distribution to a set of percentiles.

Sobol sampling

Sobol sampling is a kind of "quasi-Monte Carlo" simulation that you can select in the Uncertainty Setup dialog as an alternative to Monte Carlo and Latin hypercube. Where Latin hypercube samples more evenly than simple Monte Carlo independently from each input distribution, Sobol samples more evenly over the hyperspace of all input distributions and so reduces noise from spurious correlations. It significantly improves convergence (accuracy of resulting distribution as a function of sample size) over other methods when there are multiple (but not too many) input distributions.

International

In-place entry for PinYin input (Chinese, Japanese and Korean language typing). As you type, the PinYin letters appear directly in the text field, when you hit the space bar, the candidates popup appears directly below where you are typing.

In continental Europe, South America, and other places that use the comma-as-decimal point convention, Copy/Paste between other applications now uses the comma-as-decimal convention (according to your regional setting). You can configure this using the new DecimalForCopyPaste system variable.

Engine

Multithread computing

Analytica 5.0 takes advantage of multiple cores (processors) to speed up large array computations. Nowadays, the central processing unit (CPU) of most computers has multiple cores, each of which can run its own computations. Laptops often contain 2 to 4 cores. High-end servers may have from 12 to 60 or more cores.

Multithreaded evaluation is used in all 64-bit Analytica editions. It is not used in Analytica Free 101 or 32-bit installations, because the memory overhead mean multi-threading is rarely worthwhile in a 32-bit memory space.
In 64-bit editions, you can configure multithreaded evaluation using these system variables from menu: Definition / System Variables / Settings:
- MaxEvaluationThreads sets the maximum number of evaluation threads the engine can use during evaluation. Set this to less than your total number of cores to ensure some cores are available for other processes. This setting is saved for your computer, and persists across Analytica sessions, and across different models. It is not saved in the model, and has no effect on someone else if you share your models.
- MaxModelThreads sets the maximum threads that can be used for the current model. It is stored with this model only. You can set it to 1 if you want to keep it single-threaded for some reason.

Evaluation

You can now pass through an optional index parameter in a UDF to another function's optional index parameter. For example,
- Function Fu1( J : optional index )
  
  Function Fu2( J : optional index ) := Fu1( J )
If you omit the index when calling Fu2( ), then Fu1 is called with its index parameter omitted. This eliminates having to have two separate cases in Fu2's Definition and a test using IsNotSpecified.

Plus applied to text

The + operator should not be used to concatenate text!! Use the & operator for text concatenation. For years, Analytica has issued a warning when you use + or Sum for text concatenation. It has now become more strict. Here's a video short explaining the + concatenation error, and what to do about it.
- The standard going forward is that the result of x+y when x or y is text shall be NaN (rather than the textual concatenation). This change will enable performance speed-up opportunities going forward.
- What was formerly a warning is now a non-continuable error when encountered during a multithreaded evaluation.
- A new preference setting was added: x+y returns NAN for text. This is mostly to help deal with this change in legacy models. In some models that experience the error. Turning this on will get you through it.

Null

The null value now displays as "Null", not as "«null»".
Previously, the token Null in an expression identified the system variable with the name Null. That system variable has the value «null». Hence, there was a distinction between the system variable Null and the null value «null». Now, however, Null is directly interpreted by the parser as the null value, unless it appears in an identifier context.
- For example, suppose you have an index defined as [1,2,Null]. The third element of the IndexValue is now the null value, whereas previously it would have been a handle to the system variable Null.
- For this index, @[I=Handle(Null)] is now 0, whereas would have been 3 previously.
- Because the parameter of Handle is a object context, Handle(Null) is still a handle to the system variable, as it was previously.
- Note: An object context occurs when a function parameter is declared using an object class qualifier such as Object or Variable.
The text concatenation operator involving "Text " & Null now results in "Text Null", whereas previously the result was Null.

Function parameter qualifiers

New qualifiers for function parameters include:

Color specifies a parameter that expects a color. See Color parameters.
Module specifies a parameter that expects a module.
Class specifies a parameter that expects an object class.
OrNull allows a parameter to have the value Null, without reporting an error about the data type being wrong.

CSV and JSON files

CSV (comma-separated value) and JSON (JavaScript Object Notation) files are widely used for data interchange. New functions ParseCSV and ParseJSON make it much easier to import data in these formats. MakeCSV and MakeJSON let you generate files to export data in these formats.

New built-in functions

BitAnd, BitOr, BitXOr, BitNot, BitShift, BitCount
Canvas, CanvasContext, CanvasDrawEllipse, CanvasDrawImage, CanvasDrawLine, CanvasDrawPolygon, CanvasDrawRectangle, CanvasDrawText
ChangeNodeVisibility
CloneObjects
CloseWindow
ComparisonPart
CreateNewObject
CumBinomial, CumBinomialInv
CumPoisson, CumPoissonInv
Dawson
Gaussian { Formerly a library function }
GetArrowsOnDiagram
Faddeeva
FindObjects
Flatten
ImageFromHex
ImageInfo
IndexLength - this video reviewing how this compares to other (inferior) methods to figure out the length of an index variable
InvalidateResult
Keelin, CumKeelin, CumKeelinInv, DensKeelin, KeelinCoefficients
MakeCSV
MakeJSON
Move { as a function }
MultiChoice
OpenModelFile
OpenURL
ParseCSV
ParseJSON
ProductLog
ReadBinaryFile
Reverse
ShowAskAttribute
ShowWindow
Sleep
SobolSequence
TextDistance
TextCharacterEncode
UncertainLMH, CumUncertainLMH, CumUncertainLMHInv, DensUncertainLMH
Unflatten
VectorCrossProduct
WriteBinaryFile
WriteImageFile

Enhancements to existing built-in functions

Unique(a): Index parameter is now optional. If omitted, it returns the unique elements across all dimensions of «a».
Unique: A new optional parameter, «mapToUnique», returns a map to the first slice of «a» with the same value.
CopyIndex: Added an «omitNulls» parameter.
Gaussian now eliminates sampling error in the Covariance of the random Sample generated.
MakeDate: Added an optional «valueForInvalid» parameter.
Erf, ErfInv: Parameter can now be a complex number.
ReadImageFile: Has actually existed since Analytica 4.2, but is now documented and supported.
ParseDate: A new «dateTemplate» parameter, used to control the regional interpretation. For example, should 9/11/2016 parse as 11-Sep-2016 or 9-Nov-2016?
DatePart: Supports part codes 'EEE' and 'EEEE' for English month names.
SpreadsheetOpen: added an optional «title» parameter, allowing you to customize the caption of the open file dialog.
SpreadsheetRange & SpreadsheetCell: When the «what» parameter to these functions is 'BackColor', 'TextColor' or one of the 'Border*Color' options, the result has been changed to be an RGB integer (i.e., 0x00rrggbb in hex). Previously it was in BRG order (0x00bbggrr).
Aggregate: Added an optional «noMapError» parameter.

Images

You can now copy an image, or an array containing images, to a variable definition using the assignment operator (:=). When you view the Definition, you'll see the binary image encoded in a call to ImageFromHex. When you view the result table, you'll see the image itself.

Optimizer

You can now use a new gradient variable type, with a new node shape (a top-heavy trapezoid) for variables that compute gradients. The gradient-based solver algorithms can then make use of these analytic pre-supplied gradients during search. See Explicit gradients.
For NLPs that are encoded in a self-contained UDF or variable, expressions that compute the gradient with respect to each decision variable can be passed to the «gradient» parameter of DefineOptimization, and expressions that compute the gradients of each constraint can be passed to the «jacobian» parameter. See Gradient Parameters.
A new system variable, Sys_ComparisonTolerance, allows there to be some constraint violation before showing {!} in front of a constraint. Since solver engines allow constraints to be violated up to the 'Precision' setting (1u by default), this avoids tagging these barely-violated constraints in the result view of a constraint node.

Changes to Libraries

Multivariate Distributions

Function Gaussian is now a built-in function.
An adjustment was added to Correlate_dists and Correlate_with so that the resulting rank correlation of the samples are more accurate.

New system variables

New attributes

Example models

These example models are new:

Minimal edit distance, a 2-D dynamic: (Dynamic models) Demonstrates the use of a 2-D Dynamic, which updates across two dynamic indexes. Aside: If you need to compute edit distance, see the new TextDistance function first.
Regular polygon calculator: (Engineering) A calculator that can compute the any of the metrics of a circle, inscribed or circumscribed N-sided polygon from any of the other metrics. Illustrates OnChange events with OnChange loops.
Find Words Game: (Fun and games) See how many English words you can make out of a random tray of letters. Analytica Enterprise users get an extra treat -- click on any word (either one you enter, or one shown in the final solution) and see the definition.
Fractals everywhere: (Fun and games) Draws a Barnsley self-similar fern.
Probability assessment: (Fun and games) A trivia game of a different sort, where what counts is how well you understand your own degree of uncertainty.

ADE

General capabilities

All non-UI enhancements listed above apply equally well to ADE and are not repeated in this section.
Callbacks now allow client code to intercept function events (like MsgBox) that require user input or interaction.
CATable has greatly improved support for MultiTables.

New classes and interfaces

CAPicture

CAPicture holds an image, when an image appears in a result.

A CAPicture can be returned by CAObject::Result, CAObject::Evaluate, CATable::AtomicValue, CATable::GetDataByElements, CATable::GetDataByLabels, and CATable::GetSafeArray.
CARenderingStyle::PictureAsText, controls whether images that appear in results are returned as text (similar to "«PNG Image»" or as a CAPicture object.

CAGraphPivot

A class returned from CATable::GraphPivot that contains information about what the current pivot is for the graph of the result data is the CATable -- essentially the pivot that desktop Analytica would use if you were to view the result graph -- and provides a method for intelligently changing the pivot.

CAFiller

A class that describes the value or index that assigned to a particular graphing role. CAFiller objects are returned by methods of CAGraphPivot.

IAdeUICallbacks

A interface that has callbacks when various functions that display dialogs to the end-user are called, namely:

MsgBox()
ShowProgressBar()
AskMsgText(), AskMsgNumber() and AskMsgChoice()
File functions that ask for a file name: SpreadsheetOpen(), ReadTextFile(), ReadImageFile(), ReadBinaryFile(), SpreadsheetSave(), WriteTextFile(), WriteImageFile(), WriteBinaryFile(), etc.
ShowPdfFile()
OpenURL()

IAdeUIWindowCallbacks

An interface that contains callbacks invoked when the new ShowWindow() and CloseWindow() functions are used, and which is called when sending older typescript "Show windowType obj" and "Close windowType obj" commands. This allows the use of these in a more general fashion, including cases where scripts appear in conditionals, in functions, etc.

New properties and methods

New properties and methods to existing ADE classes are listed here.

To CAEngine

CAEngine::MaxMemoryLimit property. When set, it limits the amount of total memory usage in MegaBytes, that the ADE process is allowed to use during computations. If not set, or set to 0 there is no limit. If, during an evaluation call like ResultTable, the memory usage exceeds this value, ADE aborts the call and returns an ErrorCode for "Maximum Memory Exceeded".

To CATable

CATable::GraphPivot: Returns a CAGraphPivot object.
CATable::GetCellFormats
These new properties are used to extract information from a node that would be used by desktop Analytica (DTA) to determine how to view a table. This allows you to display a Table with ADE with the same axis' pivot as it would in DTA.

To CARenderingStyle

Enhancements to existing properties

The CAObject::DefinitionType property has a value 11=MultiChoice.
Previously, DateTime numbers were returned by ADE methods as a real number (the number of days since the date origin). They are now returned as a Date-Time type, which is thus independent of the date origin.
- The new property CARenderingStyle::DateTimeAsReal can be set to true to return a number as before.

Enhancements to AdeTest

A few enhancements to the Adetest program use the new Ade extensions.

- CAEngine::MaxMemoryLimit property has been added to the object pane
- Images that appear in results (such as in a result table) can be viewed.