From kilobyte to megabyte to gigabyte to terabyte to petabyte to exabyte -- the rise in volumes of data being used and processed in the last several decades is so great it’s really beyond our ability to easily understand.
Humans are still not really wired to intuitively process this rapid evolution.
As a result, it is hard to really think about just how far we have come, let alone what’s next. For instance, think about the 1956 IBM 350 – and for a laugh, Google the picture – which had a whopping 5MB of storage space. Now consider that the technology we use today will look equally dated in our lifetimes.
But exponential growth in storage and processing of data is also leading to a genuine difference of kind, not just of degree: it’s not that we can do the same things better; it’s that we can do entirely new things. And it isn’t just hot topics like artificial intelligence, machine learning and data matching – it’s also things that now seem commonplace, like making lending decisions or doing detailed financial reporting. What office workers spend their days doing today is very different to twenty years ago.
We’ve come a long, long way. And the quickening pace in recent times is driving massive change and transformation across organisations – and it will only accelerate.
So how to respond and what to expect?
Firstly, there are implications for people, both positive and more challenging.
Clearly, the need for data specialists is exploding. In the US last year, demand for machine-learning engineers grew twelve-fold. And as the numbers grow, new specialities are emerging, with titles and job descriptions that didn’t exist five, three, and in some cases one year ago.
This is happening not because we love titles, but because we need specialists to cope with the growth and the complexity of this new world.
Data is growing up as a function, much like has occurred with technology since the 1970s when the first Intel chips, PCs and the internet began to emerge.
Forty years ago, if someone worked in technology at a company, they probably had the word “operator” or “technician” in their title, and they weren’t part of the technology division but the technology department, which was probably part of operations, or maybe finance. And the highest-ranking technology person in the company wasn’t on the top executive team, but was two, three, or even four levels down.
Over time, of course, that all changed as technology grew in importance, departments and budgets grew, and roles professionalised. The head of technology became a CIO, there were more committees, more governance, more controls, and a higher level of operational maturity.
That’s the transformation data is about to go through.
But it’s not just specialists. There’s also growing need for “data generalists”, or the people who have a job focused on something else but need to be data-literate to succeed in dynamic economies characterised by rapid technology advances and fast-moving consumer preferences.
The bar for the required level of data literacy is increasing. The jobs of today won’t be the jobs of tomorrow and it’s very possible some existing jobs won’t be there in the future. That means being prepared to re-train, adapt and be curious. We need to become truly agile – falling into the trap of “this is how we’ve always done things” or “this is how my job has always been done” probably won’t cut it.
Likewise, businesses will have to be more adaptive and embrace cultural and technology change. They will need a strategy that simplifies, integrates and enhances the ability of a workforce to leverage technology and data innovation, which will actually increase current workers’ capability to use data, and adopt data-driven decision-making.
For those businesses that successfully make this shift, data will have a much greater presence at the executive and board table, because it’s no longer just a cog in the machine, but an entire, mission-critical machine in itself to remain one step ahead of the competition.
Think about just the past 10-15 years.
Data, in combination with other technology advances such as cloud computing, has seen young or entirely new companies, such as Google and Facebook, grow into some of the largest in the world.
No business can rest on its laurels.
Data will be crucial to future success, powering the broader transformation, supporting critical decisions, and being embedded in daily activities, regardless of industry.
The pace of change has never been faster, creating large gaps between the leading and laggard businesses.
But data governance and data quality will remain critical, requiring high levels of stewardship to ensure the underlying data is understood and maintained. Data and analytics will have to up its game on risk management, compliance, and auditability, among other things.
Just as in the 1970s and 1980s technology had to grow into its commercial and managerial maturity, data is now growing in prominence and professionalism.
It is an exciting time in the world of data — we stand on the cusp of a new way of doing business, one that will be powered by the information and analytics we have and finding increasingly powerful ways to use data.