Difference between revisions of "Retrieving Content From the Web"
Line 21: | Line 21: | ||
Near the bottom of the page that results is a link that says "Download to spreadsheet". By hovering over the link, you can see the URL that loads the CSV file for MSFT between Jan 3, 2007 and Aug 8, 2007. It is: | Near the bottom of the page that results is a link that says "Download to spreadsheet". By hovering over the link, you can see the URL that loads the CSV file for MSFT between Jan 3, 2007 and Aug 8, 2007. It is: | ||
− | [http://ichart.finance.yahoo.com/table.csv?s=MSFT&a= | + | [http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=2&b=1&c=2007&d=7&e=8&f=2007&g=d&ignore=.csv http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=0&b=1&c=2007&d=7&e=8&f=2007&g=d&ignore=.csv] |
+ | Take a few moments to study the URL, and and the format of the URL should become clear fairly quickly. The s parameter is the stock symbol, the a parameter is one less than the day number in the start date, b is the month number of the start date, etc. We need to know this format because we want our Analytica model to construct this URL for a given stock symbol, start date, and end date. | ||
= Creating [[media:ReadURL.exe | ReadURL.exe]] = | = Creating [[media:ReadURL.exe | ReadURL.exe]] = |
Revision as of 20:40, 8 August 2007
This page details a simple example of integrating external data into Analytica. In this example, historical stock data for a given stock is read from a web site and imported into an Analytica model.
The example makes use of an external program, written in C++/CLR using Microsoft Visual Studio 2005. The example thus demonstrates the integration of an external program with Analytica. The source code and steps for creating this program, ReadURL.exe, are given below. If you don't have Microsoft Visual Studio 2005, or don't wish to compile ReadURL.exe yourself, you can download it from this link: ReadURL.exe
Minimal Requirements:
- Analytica Enterprise 4.0
- .NET 2.0 framework
- Internet access (to Yahoo finance)
- Optional: Microsoft Visual Studio 2005
Getting Started
First, you need to obtain ReadURL.exe. You can either download it from the link, or you can compile it yourself using the source code provided in the section at the end of this article. You should take note of the full path of your executable. In my case, after compiling it, the executable was at:
C:\Src\ReadURL\Debug\ReadURL.exe
Historical Stock Data on Yahoo Finance
The Yahoo Finance web site provides historical stock data for free, downloadable for each stock as a CSV file. Before actually downloading the data, take a few minutes to understand how you can access the data as a CSV file.
In a web browser, go to: http://finance.yahoo.com. In the Stock Symbol box enter: MSFT (the symbol for Microsoft) and press the Get Quotes button. On the page that results, enter a start date of Jan 3, 2007 and an end date of Aug 8, 2007 and press Get Prices.
Near the bottom of the page that results is a link that says "Download to spreadsheet". By hovering over the link, you can see the URL that loads the CSV file for MSFT between Jan 3, 2007 and Aug 8, 2007. It is:
http://ichart.finance.yahoo.com/table.csv?s=MSFT&a=0&b=1&c=2007&d=7&e=8&f=2007&g=d&ignore=.csv
Take a few moments to study the URL, and and the format of the URL should become clear fairly quickly. The s parameter is the stock symbol, the a parameter is one less than the day number in the start date, b is the month number of the start date, etc. We need to know this format because we want our Analytica model to construct this URL for a given stock symbol, start date, and end date.
Creating ReadURL.exe
To read data from the web, a simple C++/CLR application was created in Microsoft Visual Studio 2005. Select File->New->Project/Solution->C++->CLR/.NET Console Application. I named my project ReadURL and placed it in C:\Src.
In the file ReadURL.cpp, I placed the following code:
// ReadURL.cpp : main project file. #include "stdafx.h" using namespace System; using namespace System::Net; using namespace System::IO; int main(array<String ^> ^args) { if (args->Length != 1) { Console::Error->WriteLine(L"Hello World"); return -1; } String^ url = args[0]; WebRequest^ request = WebRequest::Create(url); WebResponse^ response = nullptr; StreamReader^ reader = nullptr; try { response = request->GetResponse(); Stream^ stream = response->GetResponseStream(); reader = gcnew StreamReader(stream,Text::Encoding::UTF8); Console::Write(reader->ReadToEnd()); } catch (WebException^ x) { Console::Error->WriteLine(x->Message); return -2; } if (response!=nullptr) response->Close(); if (reader!=nullptr) reader->Close(); return 0; }
This program accepts one command line parameter, a URL, and then reads from that URL and echos the text to StdOut.
Enable comment auto-refresher