Retrieving Content From the Web

This page details a simple example of integrating external data into Analytica. In this example, historical stock data for a given stock is read from a web site and imported into an Analytica model.

The example makes use of an external program, written in C++/CLR using Microsoft Visual Studio 2005. The example thus demonstrates the integration of an external program with Analytica. The source code and steps for creating this program, ReadURL.exe, are given below. If you don't have Microsoft Visual Studio 2005, or don't wish to compile ReadURL.exe yourself, you can download it from this link: ReadURL.exe

Minimal Requirements:

• Analytica Enterprise 4.0
• .NET 2.0 framework
• Internet access (to Yahoo finance)
• Optional: Microsoft Visual Studio 2005

Getting Started

First, you need to obtain ReadURL.exe. You can either download it from the link, or you can compile it yourself using the source code provided in the section at the end of this article. You should take note of the full path of your executable. In my case, after compiling it, the executable was at:

C:\Src\ReadURL\Debug\ReadURL.exe

Historical Stock Data on Yahoo Finance

The Yahoo Finance web site provides historical stock data for free, downloadable for each stock as a CSV file. Before actually downloading the data, take a few minutes to understand how you can access the data as a CSV file.

In a web browser, go to: http://finance.yahoo.com. In the Stock Symbol box enter: MSFT (the symbol for Microsoft) and press the Get Quotes button. On the page that results, enter a start date of Jan 3, 2007 and an end date of Aug 8, 2007 and press Get Prices.

Near the bottom of the page that results is a link that says "Download to spreadsheet". By hovering over the link, you can see the URL that loads the CSV file for MSFT between Jan 3, 2007 and Aug 8, 2007. It is:

http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=0&b=1&c=2007&d=7&e=8&f=2007&g=d&ignore=.csv

Creating ReadURL.exe

To read data from the web, a simple C++/CLR application was created in Microsoft Visual Studio 2005. Select File->New->Project/Solution->C++->CLR/.NET Console Application. I named my project ReadURL and placed it in C:\Src.

In the file ReadURL.cpp, I placed the following code:

// ReadURL.cpp : main project file.

#include "stdafx.h"

using namespace System;
using namespace System::Net;
using namespace System::IO;

int main(array<String ^> ^args)
{
	if (args->Length != 1) {
		Console::Error->WriteLine(L"Hello World");
		return -1;
	}

	String^ url = args[0];
	WebRequest^ request = WebRequest::Create(url);
	WebResponse^ response = nullptr;
	StreamReader^ reader = nullptr;
	try {
		response = request->GetResponse();
		Stream^ stream = response->GetResponseStream();
		reader = gcnew StreamReader(stream,Text::Encoding::UTF8);
		Console::Write(reader->ReadToEnd());

	} catch (WebException^ x) {
		Console::Error->WriteLine(x->Message);
		return -2;
	}

	if (response!=nullptr) response->Close();
	if (reader!=nullptr)   reader->Close();

	return 0;
}

This program accepts one command line parameter, a URL, and then reads from that URL and echos the text to StdOut.