Big data without the big headaches: How to get your strategy right
Your big data project doesn't have to be a headache - if you get your strategy right.
Big data is big business: analyst Gartner says the amount of chief data officers (CDOs) being appointed by major organisations rose from 400 in 2014 to 1,000 in 2015. The analyst predicts 90 per cent of large companies will have a CDO by 2019.
Such CDOs can help firms to focus on the value of analytical information. But all organisations -- with or without the appointment of a CDO -- must find ways to demonstrate the value of big data, and that job often falls to the CIO.
So when it comes to setting a big data strategy, what is the role of the CIO and how can he or she help the rest of the organisation to make the most of their information? ZDNet speaks to the experts and discovers best practice advice.
1. Get access to the right kind of talent
The first thing to recognise, says Camden Council interim CIO Omid Shiraji, is that the demand for information management is only going in one direction: up. One coping strategy is to draw on external talent.
"As a CIO, you'll never know everything about data," he says. "But you'll have to make sure you surround yourself with people who do. CIOs will need to recognise the talent, to relate those skills to the broader organisation and to create an environment to allow smart people to be successful."
The focus on analytics has a knock-on effect on the labour market. Demand for big data specialists is set to rise 160 per cent through to 2020, according to SAS and the Tech Partnership. Estimates suggest there will be around 56,000 job opportunities per annum by the end of the decade.
Finding great analysts, however, is just the starting point. Firms must be careful to ensure the insight they create is untainted. Andrew Marks, former CIO and now the UK and Ireland managing director for energy in Accenture Technology Strategy, raises the issue of data specialists being affected by confirmation bias.
"Such people only look for patterns they expect to see," he says, suggesting some scientists look to fit information to an expected model. "It's important to understand that access to lots of eyes can produce different results. Big data gives businesses the opportunity to look at an issue from many more viewpoints than was possible in the past."
2. Prove the value of big data to the business
A great data strategy, therefore, is crucial. Firms need to think very carefully about how they integrate data sources and how they make sense of the relationships that emerge through analysis. IT leaders will play a crucial role, as the CIO's experience in leading big business intelligence projects means he or she is often rightly viewed as the technology expert within the business.
CIOs must rise to the challenge and work with their line-of-business peers to create valuable insights in key areas, such as experience and performance. First Utility CIO Bill Wilkins is one IT leader who has worked with his peers to use analytics to help the organisation transform.
First Utility started as a small, entrepreneurial business in 2008 and experienced rapid growth. Wilkins joined First Utility full-time in 2010 and saw the potential for big data to help the firm meets its long-term aims. "The investors were clear at the inception of the company that they wanted to disrupt the market," he says.
"They wanted customers to save money by giving them the best price tariff and they wanted them to save more money by giving them information to think carefully about their energy use. We started that process seven years ago; we're better at it now and we'll be even better in another 24 months."
"By having a combined organisational structure in place for architecture and data, it's been easier to see misalignment," says Wilkins. "We're continually trying to create a forensic take on our data in an attempt to understand the information the business holds in more detail."
3. Take advantage of the democratisation of data science
Mark Ridley, director of technology at reed.co.uk, is another IT leader who is focused on the value of big data. He is currently helping his organisation to build its data science team. Ridley wants to find ways to use that skilled expertise, and the analytical information these individuals produce, to help the organisation function more effectively.
Ridley says there have been huge advances in regards to democratising data science. He believes a tool like Apache Spark makes machine learning far more accessible. Ridley says firms can then use another product, such as Data Bricks, to run a dashboard-like interface on top of its Spark clusters.
"The type of analytics that used to be quite difficult is now being embedded within your systems," he says. "Our data science team are huge Spark evangelists. Taking that information, and plugging it into our business intelligence system and data warehouse, is a huge focus for us as a business."
Ridley says Reed.co.uk can use information from millions of CVs to make smarter recruitment recommendations to candidates and companies. Internally, the application of data allows Ridley and his business peers to become more proactive. They can see when new trends in recruitment are about to emerge, both at a sector level and within individual organisations.
"There's a huge array of things we can potentially do," says Ridley. "It just means we have to be able to draw together lots of data from many different systems. What we're trying to do isn't ground breaking in theory, but making that approach work operationally is the biggest challenge."
Read more essential CXO and business leadership stories
- DevOps enters the mainstream: Here's how to make it work for you
- Best practice advice for moving to the cloud
- Five best practice tips for investigating the IoT
- Big salaries, bigger challenges: How to hire and keep the best tech staff
- Less politics, more action: How CIOs can benefit from taking an interim role
- The lessons of the cloud: What have we learned so far?
- Whatever happened to green IT?
- When to talk, when to shut up: How honest should you be with your peers?