Data engineering Making your data work better for you

Optimising data lakes for financial services

Discover four practical ways (and a warning) to improve the performance of data lakes in your organisation

Read our latest white paper
Data engineering Data engineering

In your organisation, data matters. Your business works in an intricate industry, and you’re under constant pressure to be flexible, resilient and yet innovative. We can advise on the right architecture for you, and work with you to create it.

Scott Logic works with industry-leading clients with high ambitions and complex systems. We can offer support on building bespoke solutions, or guide you to the best open source or commercial products for your purposes. We deliver solutions that respond to business needs, from accelerating data flows to optimising data storage and retrieval.

What we

At Scott Logic, we deliver our data engineering projects based on firm principles:

At Scott Logic, we deliver our data engineering projects based on firm principles:

Listen, and challenge

We are a company that believes in learning, and challenging assumptions. Our clients respect us because we know when to listen, and when to push for something transformative.

Best fit, not biggest headlines

We use our broad experience of tools and technologies to select the best possible option for a client, not just the new release with the big fanfare.

Knowledge should be shared

We’ve been here time and time again. Our insight allows us to share what works well, and explain potential pitfalls.

Each project is unique

Businesses have different ideas, ambitions and requirements, even in the same industries. We take the time to get to know what a client needs before offering suggestions.

How we can
help you

Whether you’re building a bespoke solution, or seeking advice on an existing system, we can assist in various ways

Delivering effective data lakes

Data lakes offer exciting new possibilities for your business, and could have a significant impact on areas such as reporting and analytics. We can advise you on how data lakes can help address your challenges, and which design is best for you

Adopting single source architecture

We can help you choose and build architecture to tackle a range of issues. With the optimum solution, you can reduce duplicated or conflicting data, simplify data flows, and fix data quality issues with less effort.

Moving from batch to streaming

As streaming technologies become more advanced, we can help you simplifying your operations and reduce maintenance by making the switch. Whether you are looking to improve latency, processing or ease of operation, we can help you take the right path.

Building scalable, resilient systems

With so many different approaches on the market, it can be challenging to find a solution that meets your needs. We suggest technology that delivers on your requirements, and help you tailor it to your business.

For more information, contact our Head of Data Engineering, Andrew Carr.

Are you looking for ways to make your data lake more effective?

Read our white paper on Optimising Data Lakes for Financial Services

Read more
From the

Chess data mining with Apache Spark and Lichess

Chess data mining with Apache Spark and Lichess

Lichess makes over 100GB of chess games from 2017 available on their website. This post shows how this data can be transformed with Apache Spark and analysed. Something for Data Engineers and Chess Enthusiasts alike!

Cassandra - achieving high availability while maintaining consistency

Cassandra - achieving high availability while maintaining consistency

A discussion about Cassandra consistency levels and replication factor, which are frequently misunderstood. This post explains the Cassandra infrastructure and how its configuration can be tuned.


If you like working in smart teams on challenging software projects,

Check out our current vacancies