Spotlight: Improving London-wide planning data

Our latest Spotlight is a guest post from authors Molly Strauss, Principal Policy and Programme Officer – Growth and Infrastructure; Peter Kemp, Planning Change Manager; and Theo Blackwell, Chief Digital Officer at the Great London Authority. As part of the Mayor’s Smart London work, efforts are underway to improve the quality and quantity of planning data, boosting efficiency, reducing administrative burdens and moving towards a ‘live hub’,  accessible to all Londoners.

Planning is ‘core business’ for the Greater London Authority, and so Chief Digital Officer Theo Blackwell has initiated a London-wide collaboration to streamline how data on coming development passes between London’s 35 council planning authorities and City Hall.

We began the project by conducting research to understand the particular challenges of gathering London-wide planning data. Findings also give insights into wider issue identified in the Smarter London Together Listening Tour – namely fragmented approaches and IT systems across London government.

The London Development Database

Currently, council planning authorities are required to submit data via the London Development Database (LDD), focusing on new dwellings that have been given permission, commenced development and their completions. For commercial development, we collect information about major changes of use and major industrial development (over 1000sqm) and loss of public open space. Councils are expected to add permissions monthly, within three months of the permission being granted. Information on when work starts or is completed is updated annually.

Established in 2004, the LDD remains an essential source of information for the Mayor to monitor delivery of the London Plan. However, it produces an incomplete picture of development in London since it doesn’t include information on pipeline development  (projects not yet approved) or schemes that are refused.

London Development Data


We first set out to understand business-as-usual at councils before developing a plan to improve the LDD.

From February-May 2018, we travelled to 26 council offices  so that we could speak with wide-ranging groups including planners, development managers, monitoring officers and ICT.

IT Systems

Planning authorities face constraints on all sides — including using legacy systems and processes.

Many authorities are frustrated with the systems they use to track planning applications, which are expensive to change and offer limited functionality. System fields frequently do not reflect planning authorities’ needs nor the LDD requirements. Proper integration between these systems and authorities’ other software — or even modules created by the same provider — is consistently a challenge.

Only 21 percent of authorities are satisfied with their back office systems, and half of those we spoke with are considering or actively pursuing procurement.

On average, 79 percent of planning applications enter authorities via Planning Portal, a webtool created as a joint venture between MHCLG and a private company. Planning Portal automatically populates most authorities’ systems with information on each application. However, because Planning Portal’s fields do not fully match authorities’ needs nor the LDD requirements — and because many applicants do not populate the existing fields correctly — only very basic information is pulled through into planning authorities’ systems.

For many authorities, information required for the LDD is not entered into development management systems. Therefore, officers must trawl through documents manually, hunting for the figures they need. Once the information is located, data entry into the LDD is still largely manual.

development management systems and suppliers
Local authorities use different suppliers, and within these choose different development management systems – making integration a significant challenge. Of authorities surveyed, only 21% were happy with their systems, half were considering or actively pursuing procurement.

Time taken by councils to complete LDD requirements

Completing the LDD takes on average 60 hours monthly—costing London’s councils approximately £750,000. Starts and completions are particularly difficult to track, because building control records are often unreliable. Officers piece together this data through a variety of sources, from street naming and numbering to Google Streetview images to site visits.

To complicate matters, the details of a scheme can change between application submission and permission — so council officers cannot rely only on original application documents for source data.

Teams are sometimes unaware of useful data held by their colleagues, since this information is distributed across systems, within disparate spreadsheets, or collected inconsistently.

Different approaches

Providing information to the LDD is a requirement but London’s planning authorities approach it in different ways, leading to variation in the quality of information gathered.

For some, the LDD is the only source of monitoring data for development, feeding directly into Annual Monitoring Reports and housing trajectories. For others that have their own separate monitoring systems, this can lead to potential duplication.

Our Vision

Across planning authorities, officers and managers know that the LDD process could be more efficient and many are eager for change. Our findings helped us develop a plan to improve efficiencies and lower the administrative burden planning authorities face while also increasing both the quantity and quality of development data openly available to the public sector, businesses, and Londoners.

Our ambition is to create a ‘live hub’ of planning and development information, accessible to all Londoners by reforming the information we collect and the way we collect it.

We will request the data we need for monitoring up front, on the initial planning application. This data will pass seamlessly into authorities’ improved back-office systems, where planners will verify it, and then automatically out of those systems to City Hall and onto a public website.

This will require some changes. First, the burden for filling out information should be pushed to the applicant (the developer) not the local authority. Secondly, as the market need changes the current crop of IT suppliers of planning systems will have to change their products to be more responsive.

A new live hub would include information currently collected by the LDD, but also much more — including a feed of planning application data from all 35 planning authorities.


Automation will lower the burden on councils and free up resources. But we expect this project to deliver benefits beyond that, including:

If we are successful, we can pass on our learning to other cities and regions across the UK looking to digitise planning.

Realising our vision will require changes to the whole development data journey. Follow Smart London on Medium to learn more about how we plan to tackle this.