GI => GO (V) Data Validation

Thanks to a great old post on the TradingBlox Roundtable Forum (by the inimitable “Sluggo”), I will add some new tests to my data validation protocol. This post is part of a series including:

1. Garbage In => Garbage Out
2. Creating Good Price Series
3. Data Validation
4. Generalized Extreme Value Distribution

New tests

• Low <= Open <= High
• Low <= Close <= High
• Intraday range within expectations (Use Extreme Value Distribution).
• Unusual number of days without price data.
• Final date in series matches expected value.
• OHLC match legitimate tick values.

Old tests

• Most recent value is “reasonable”.
• Change in raw close on each roll-over is within expectations.
• Daily change in raw close is within expectations (Use EVD).
• Difference between raw close and Panama close is constant between roll-overs.

One particularly nice idea Sluggo had was to run the validation from within TradingBlox itself – I use an R script. I use a file of previous “resolutions” to save highlighting old problems and I write a new file each time as a “Quality Record”. This can be handled using Blox Basic. So, it seems that I could easily port this process into TBB!