Algorithmic Trading using Complex Event Processing (CEP) and Data Mining
Algorithmic Trading is automated trading done by computer programs (algorithms) that react to patterns in financial market data or financial news. As of 2009, high frequency trading firms accounted for 73% of all US equity trading volume.
Several different strategies are used for algorithmic trading. For example index arbitrages exploit differences between pricing of indexes and the stocks they represent. Market making is another strategy that aims at earning money on the spread between bid and ask price. More complicated strategies use data mining and pattern recognition.
Personally I believe there's much truth in the saying that the stock market is mostly psychology. How do you believe the following influences the stock market?
- Dividends and issues of shares. Overreactions.
- Buy- or sell recommendations from analysts.
-
Publicity in newspapers.
-
Weather.
-
Season. Report periods.
-
Political events.
Getting Started
What do you need for your do-it-yourself Algorithmic Trading system?
-
A high quality data source of market data.
-
Reliable sources for other data.
-
An automated financial instrument order handling system.
-
A CEP (Complex Event Processing) engine.
-
A data mining engine.
Financial data feeds are available as standard services from several financial companies. There is even a standard protocol for the information, FIX (Financial Information eXchange). Also large financial news bureaus can supply their news in a format that is adapted for automatic analysis.
Financial brokers exists that can handle your orders automatically at decent commissions (of course depending on your trading volume). You should be able to find them through your internet stock broker, which may supply this service themself or have a partner that does.
Complex Event Processing and Data Mining are supported by SQL Server 2008 R2 through StreamInsight and Analysis Services. You can learn more about StreamInsight in my articles on SQLServerCentral.
Evaluating your algorithms
Some factors to consider:
-
"Edge" (Return On Investment in the long run)
-
Opportunity (how often will there be any action)
-
Accuracy (how often will there be a win)
Personally I'd prefer an algorithm that gives a stable income compared to a high risk algorithm.
Risks
Algorithmic Trading, like any other investment activity, requires you to be cautious. A friend told me about his neighbour that earned millions one year and lost them (and his house) the next. Like in poker you have to decide on a maximum loss level where you are ready to take the pain and leave.
On the other hand, Complex Event Processing is also used by trading firms to reduce their risk. It can help you monitor your risk exposure in real time.
Also, very important... just that you have found a pattern that worked historically doesn't imply that it will work in the future. Have you heard about "The Bible Code" (or Torah Code) which describes encrypted messages in the bible? Only... you could find encrypted messages in any book if you search long enough. Remember, that is the weakness of Data Mining.
My recommendation is to keep to a test environment until you are sure about your algorithms. Also it is not a bad idea to take a course in statistics to learn how to make statistical significance tests.