Scott Logic worked with a major international investment bank to produce a powerful tool for analysing price correlations across asset classes.


The financial analysts working for one of our clients required a means of analysing price correlations across 120 different economic assets, grouped into 12 categories of 10 instruments, ranging from FX prices through various indices to bonds and asset classes. A tabular approach would require the analyst to pick out the significant correlations from a table containing over 14,000 values, clearly a challenging proposition. We concluded that the only viable approach was to generate a visualisation of the various correlation values, something in which we excel.


  • Java (Servlets, JSPs, JSTL)
  • JavaScript (JQuery)
  • Sybase


After discussing the requirements with the client, we decided the best solution was a heatmap, a powerful tool which allows the user to quickly identify variables that display significant correlation.

In addition to visualising the raw data, our application also allowed the user was to manipulate the data in a variety of ways, for example using normalised or moving average values. In total there were almost 1500 combinations of input parameters and almost 20 million discrete values across all the different combinations, leading to a dramatic expansion in the number of calculations. It was therefore no longer feasible to generate the visualisation in real-time.

The initial solution took almost 100 hours to build the full results set, which meant that the information would be out of date before it could be used. Some optimisation was obviously called for.


Our approach to reducing the time taken to generate the visualisation included using three levels of caching to pick out common calculations, which reduced the time for the entire results set to be calculated from 100 hours down to 20 minutes. The calculations were carried out overnight, triggered by a scheduled job, to ensure that the data was available for the analysts to view as soon as they got to their desks in the morning.

The display of such large data sets also caused problems that we needed to solve. The initial approach utilised an HTML table to display the data but various browsers could not render this quickly enough to provide an acceptable user experience. We resolved this by rendering the heatmap as a 12×12 grid of images that were dynamically generated server side. We added a checksum to the image URL to ensure that every image within the view was based on the same snapshot of data.

The user could zoom into a single image which would then be rendered as a table with extra information displayed on hover over individual cells. The user could also click through from a cell to a chart of the correlation time series.