Industry Insights

Big Data in the Big Apple

A few years ago Michael Flowers worked as a lawyer in Iraq for the Department of Justice. These days he has traded Baghdad for New York and has become a well known figure in the world of Big Data and business intelligence software. As New York’s first Director of Analytics, he was tasked by Mayor Bloomberg with helping city officials tackle outstanding problems that were proving difficult to resolve.

Transforming New York

These issues included helping the police discover illegal house conversions and cracking down on illegal dumping of cooking oil into the sewers. The former previously relied on guesswork and anecdotal information, but by looking for a correlation between several data objects including brickwork maintenance, rat sightings, fire incidents and tax payments, they were able to concentrate the inspectors’ time on the most likely areas. This dramatically increased the success rate of finding illegal conversions from 13% to 70%. There are approximately 20,000 licensed restaurants in New York and each must have a license with a regulated hauler to take away the grease they generate. By pulling together data from other city departments, including the Business Integrity Commission which is responsible for the haulage license, inspectors quickly identified restaurants that were operating illegally, which led to an astonishing 95% success rate.

Office of Policy and Strategy Planning

Left unattended, both of these examples have significant cumulative effects, which in turn lead to more problems for other civil departments and increased city costs. In the current economic climate, budgets are constantly being scrutinised and departments are always looking for ways to improve their efficiency. Michael Flowers and New York’s office of Policy and Strategy Planning is helping officials tackle long-standing problems by analysing complex data, but their success would not be achievable without good quality data, and an effective means of translating vast amounts of data into something easy to digest. This has allowed city officials to make better decisions built on evidence-based analysis. Gathering large amounts of data and then looking for answers will not help any business or organisation. They need effective analysis tools, such as business intelligence software, which allows an organisation to construct charts based on explicit data queries. In New York, city officials had very specific questions in mind and then went in search of answers.

Business Intelligence Software

Business Intelligence Software is very good at visually displaying data in a way that is easy to identify activity outside the norm, but more importantly it can be used to help pinpoint the cause of such behaviour. If there is a particular problem in your business a business intelligence dashboard will allow you to build a query and then drill through layers of information to identify the cause. Equally, business intelligence software can be used to model success and investigate if any factors are transferable to other parts of the business. On a day to day basis, the software can be used to pre-empt a situation before it develops into a significant issue. The success in New York is not unique and the lessons learnt can be applied to any business that is looking to improve internal efficiencies and tackle long-standing issues.

Written by Ken

Featured resources