All articles

Silos are architectural heritage from the pre-data era: it is time to move on

Originally published in the Jan / Feb 2017 Edition of the Public Sector Focus Magazine (pages 26/27)


I left University in 2000 to start a career in the public sector. In that first job having an external email address was the exception, and sharing of data was hardly straightforward. We’ve come a long way since then. Organisations are seeing the availability and variety of data revolutionise the way that they operate. Data driven businesses value decisions that are supported by verifiable data. At the same time, large organisations are often far from nimble and so there is a lot of variability in practice.

One of the biggest changes has been government organisations recognising that working in isolation makes achieving things harder. This has been a factor in supporting the huge increase in Open Data publications.

Any public sector organisation sits within an ecosystem of partners, stakeholders and the wider community. Thinking that data can mainly flow in one direction – to that organisation with command and control – just doesn’t work.

There are, however, public sector organisations that are evolving their operating models. They recognise that they can play a more effective role if they are designed in a way that acknowledges that they don’t exist in isolation. Data requirements and data flows are a core part of these new organisational designs. These organisations recognise that they are part of a connected data ecosystem.

One of the organisations on that path is the Food Standards Agency (FSA). The role of this non-ministerial Government department is to use their expertise and influence so that we can trust that the food we “buy and eat is safe and honest”. The challenge is that the food chain is large and complex, and has a huge variety of people, businesses and other organisations involved.

Data is key for all parts of the FSA’s roles and responsibilities. However, the FSA can’t be the creator, owner or user of all that data. It needs to be part of a connected data ecosystem. What role the FSA plays within that ecosystem is the challenge and opportunity that they and other public sector organisations are beginning to explore.

The FSA have made strong open data commitments and they have published a strategic plan with data as a key enabler and driver of change. They are developing an understanding of what data may be needed, how it will be collected and flow, and what role other organisations will play in the new operating model.

In such a connected data ecosystem, data needs to be accessible and not locked in silos. Developing data management strategies that are resilient and adaptive to change is also a key factor.

There are building blocks required to enable collaboration across organisations, including:

  • open standards – to reduce lock in and aid portability, open standards can reduce the need for all organisations to be reliant on the same technology or a single vendor
  • providing connectors that help data to be easily accessible – data from lots of different sources can’t all be stored in one huge data warehouse


workshopping data concepts with the Food Standards Agency
Workshopping data ideas and a future connected data ecosystem with the Food Standards Agency

For example as the FSA supports food business operators, it will need to find and develop common standards and identifiers. Data flows lose friction and gain velocity when we all use a common set of terms for the same concepts. This will help in designing approaches that are flexible and not built around traditional silos. This will help the FSA to focus its efforts where they have the greatest effect, and to support businesses in providing data in ways that minimise any burdens.

Luckily, there are already technologies designed to help connect data together at scale. The concept of a web of data that relies on a set of open standards is crucial in meeting this challenge. Linked-data or semantic technology is built on open standards of the World Wide Web, rather than the silos of proprietary systems or single domains. It gives organisations a way of wrestling data out of legacy systems and connecting it together to solve problems.

Linked Data doesn’t just connect datasets, but links individual pieces of information or statements such as people, places and organisations. It can mean that data at different levels can be shared and used.

Linked-data turns ten this year. The technology was originally developed as part of a W3C initiative called the Semantic Web, with a mission “to liberate data from the tyranny of applications”.

At Epimorphics, we’ve been working with many public and private sector organisations in building data that they and others can use. Significant benefits are being realised as organisations are able to tap into this web of data. The next step is identifying data from across the public sector that, when made available in the same way, will have the most impact.

For the Food Standards Agency this means understanding the opportunities new data approaches can help solve. This could be publishing reference data and standards, such as food-type coding, to aid identification in other people’s data. Or it could mean stopping creating data because there are others who do it better. Sometimes it means redesigning a dataset so that it is easier to use and reuse. All these approaches are aided by recognising data as an asset, making it accessible, and helping it connect to other data.