We are currently working across a number of roles requiring the elusive ‘big data’ skills so thought it was worth sharing some views from a search perspective. The talent exists and is accessible but it is our impression that recruitment processes are collapsing through confusion and misalignment of skill sets. So how should we be thinking about big data in the context of talent acquisition?
The market is white hot and has clearly matured to a point where everyone in the boardroom wants to be talking about data and its meaning. Ironically given all the clarity ‘big data’ promises to provide, the talent market is abundant with smoke and mirrors when it comes to knowing ‘who does what well?’ It is our view that as a ‘layman’ (99% of head-hunters in this sector should be classified as relative laymen given the immaturity and inherent complexity of the area) we should be looking to work where possible in absolutes. Where not available we need to drill down to a level of granular detail that is manageable. Is a terabyte a lot of data? Is a 100 terabytes a lot of data? Is it difficult to collate and analyse video data, twitter data, text messages, voicemails, blogs? We all know and can draw meaning from the three Vs (Variety, Volume, and Velocity) often used to identify big data. However what if the candidate has been dealing in two V’s not three or one V was truly gargantuan and challenging but the others were not?
Going forward I think we can expect to see a real delineation of skills and positions, with three key strands emerging:
- Build: big data needs big architectures and platforms before collection, storage and interrogation can begin.
- Find and clean: poor quality data or simply not enough is the primary enemy of most successful big data projects. This is probably the least developed area of the big data discussion as significant uncertainty remains - what is enough data to make a ‘good’ decision?
- Derive meaning: Once the platform is built and the data collected it falls to those gifted enough to do the analysis and derive some tangible meaning. They will be responsible for peering into the crystal ball and discovering previously hidden approaches. Their suggestions should come with degrees of certainty sure to please business leaders worldwide. Invariably it will be these people that rise to the top as the future rock stars of the big data profession.
The above delineation is not a revelation but I think it’s helpful to think of the talent within the market as ‘strands’, as opposed to the broadly useless requirement for ‘big data skills’. It is also worth noting that very few people (if any) are capable of working effectively across the three strands.
Fundamentally we need to deal in what we can understand and log information that we don’t. It is the role of the hiring managers and stakeholders to work very closely with their chosen headhunting partners to share as much tangible information as is possible. An example could be ‘our organisation currently collates 1 terabyte of unstructured data per minute of which we currently analyse 5%’.
This is a powerful piece of information because it allows a repeatable query to assess candidate’s skills against.
When it comes to accessing candidates responsible for ‘deriving meaning’, outcomes are really the only useful currency available to a layman. These need to be framed and qualified robustly by candidates. The market is now mature enough that ‘blue sky’ answers are not meeting expectations. Realistically outcome based information should be at the fingertips of any credible candidate in the sector.
The excellent news for all involved is that the market is booming; good ‘big data’ skills are rightly in high demand and increasingly well paid. On the other side we are hearing that companies are truly starting to see significant business benefits from employing the skills. Hopefully we can all work together to ensure the right people are sitting in the right seats!
Callum Wallace is a Consultant in the Technology Practice at Berwick Partners (an Odgers Berndtson company) actively recruiting innovators, disruptors and pragmatists in the Big Data arena.