Business Strategy
CIO
Data Management
Data Professionals

The CDO Matters Podcast Episode 42

Top Data Predictions for 2024

X

Episode Overview:

Are you a data professional interested in better understanding what the top trends that could affect your work will be in 2024? What is the future of data products? How will AI influence data governance and data management?

Malcolm covers these topics and many, many more as he shares his perspectives on the direction of the industry of data and analytics in the coming year.

Episode Links & Resources:

Good morning, afternoon, or evening, whatever time it is, where you are, wherever you are. My name is Malcolm Hawker. I’m the host of the CDO Matters podcast.

Thanks for joining today. We’re gonna do a little bit of a, forward look on twenty twenty four. I know these things are very popular. As part of the new year, you look back and you look forward, but that’s kinda what new years are about.

Right? You take stock in the year that is big and you look forward to the year that will be. I’ve seen a lot of people online kind of pooh poohing this idea of, you know, like, don’t look forward. It’s kinda silly.

It’s like, I like this stuff. I like their retrospectives as you would have seen in two episodes ago where I summarized some of the key trends in the data and analytics world twenty twenty three.

And I will do the forward look on twenty twenty four now, today, because I like this stuff.

In case you’re new to the CDO Matters podcast, I suspect some of you will be, and I thank you for tuning in. This is the podcast for chief data officers, vice presidents of data and analytics, directors of data and analytics, managers of data functions, data stewards, data owners, data governors, you name it.

If you are in the field of data, data and analytics, business intelligence, artificial intelligence, maybe even, if you live and breathe in the world of data and you were interested in learning more from somebody who’s been around the block and who talks to some pretty compelling guests on a regular basis every two weeks, then you’ve come to the right spot. We talk about all things data with a special lens on the chief data officer because, honestly, guys, that’s kind of the top of the food chain when it comes to people who are on our career paths.

So my thanks to you for tuning in. My thanks to you for watching. If you’re catching this on YouTube, this will be our forty second episode. That is amazing to me. Thank you for your loyalty. Thank you for listening. Thank you for watching.

It’s been it’s been a really fun ride. I never thought that I would be the host of a podcast. I never thought I would be doing this.

But it was kind of a natural extension. I spent three years as an analyst at a little firm called Gartner where I wasn’t when I wasn’t building magic quadrants and defining hype cycles.

We’re gonna talk more about hype today. When I wasn’t doing all that stuff, I was talking to CIOs and CDOs, all day, every day. So I think this is just kind of a natural extension of that. I spent three years having great conversations about best practices, the do’s, the don’ts, building strategies, executing against governance frameworks, staffing data and analytics teams, you name it. I talked about it.

And now I do that here on Spotify and Apple and Google Podcasts and YouTube, and I share what I know.

So that’s the podcast.

Today, we’re gonna talk about twenty twenty four, what I see coming. Stay tuned to the end, folks. This is the teaser.

Maybe I’m learning to become a little more media savvy. This is the teaser for kind of, like, my long shot prediction of the year, which many of you may find is a little overly doom and gloom, perhaps.

That remains to be seen, but I’ve got a bit of a long shot prediction because I’ve been doing a lot of studying. I’ve been doing some reading over the holidays as I suspect many of us have.

And that allowed me to come up with this just this total hail Mary that, is related to AI. And I’m kind of excited to talk about it a little bit because I because I think there there needs to be a little bit more talking about what I’m gonna share with my long shot prediction.

Alright. Any other kind of logistics housekeeping issues? I’d like to cover these things, off at at the beginning, and we’ll dive right into the Conteil right away.

I suspect you’ll be watching this in the last week of January. If you are and you are in the amazing, weird, wonderful city of Austin, Texas, come out to Day to Day Texas, January twenty seventh.

It’s at the AT and T Center on the campus of the amazing University of Texas. If you haven’t if you live in Austin, you haven’t been to that facility, it’s I think it’s off MLK, which I think is fifteenth. Is it not? Anyway, if you live in Austin, you know what I’m talk y’all know what I’m talking about.

If you haven’t been at that facility, it’s a great facility. If you haven’t been at the event, which I have not, I’m totally excited. Come join me. I’ll be presenting on data products at Data Day Texas.

So many other amazing speakers will be there. So many other fun people in the data and analytics space will be there. It’s on a Saturday, which is a little bit of a a different twist, which is very, very cool. But I’m I’m totally looking forward to day to day texts on Saturday.

The very day that you are more than likely consuming this, perhaps, I will be hosting a data governance forum in Brussels, which is awesome. We are expecting between seventy five and a hundred CDOs and other senior data leaders from across Benelux and all of Europe to be joining myself and some friends, from, Microsoft and from Avanade and a few other wonderful organizations that are supporting this to be talking about things like data governance, to be talking about things like the data fabric, to be talking about things like the future of master data management. So if you’re in Brussels, and the event hasn’t already happened, I think it actually may be the very day that we release this, so it may be it may be too late. But I look forward to bringing those insights back.

This is another thing that I do in this podcast that if you haven’t figured this out by now, and you haven’t tuned in before, I I literally I I live such a charmed life.

Guys, I get to travel the world and go to data events and talk to CDOs and CIOs and data leaders and thought leaders and pundits and vendors and you name it. I talk to everybody because that’s what I do.

And then I bring all that back here and I share it with you. So if you don’t get to go to day to day Texas or to data governance forums in Brussels, like, I get to go to.

Brussels is an amazing city, by the way, if you’ve never been.

Holy and completely and totally recommend Brussels. Base one of the kind of unsung gems, I think, of of of Europe. Not a lot of people go there, but you should.

It’s awesome. Anyway, I digress.

I get to go to these amazing places all around the world, and I get to share them with you. If you don’t get to go, tune into the podcast because I’m gonna give you insights from these data conferences that I go to.

All the big ones, generally, I go to. Looking forward to after January, looking forward to right around the corner in March is the Gartner Gated Analytics Summit in Orlando.

That’s kind of the big daddy, the grand daddy, maybe. That’s the phrase he used for the roast bull here in the United States.

The granddaddy of the bowl. I don’t know if it’s appropriate to say that for a gardener or a day to some, but it’s a big one. I mean, a lot of people go. A a lot of fairly influential people go and anyway, I will look forward to summarizing that and and and so many more data events through the year.

So it’s it’s part of one of the one of the things I love most about the job is I get to speak.

I get to talk to people, and then I get to share what I know.

And that’s wonderful, and I get to do it on this podcast, which is which is even better.

I can’t even believe that I have this form. It’s it’s fantastic.

I know anybody can create a podcast, but, I am blessed, that I do have, people who support me.

I have have great folks at the Prostate team. Manisha is a huge help with this. Ben has been a huge help for a long time. We’ve even got folks folks that are helping with with editing, a gentleman named Ivan. Thank you, Ivan. I’ve never met you. I look forward to one day meeting you.

But so many people that are helping, putting this all together. I couldn’t do without them, and I should have noted them in my twenty twenty three retrospective. Thank you.

Gracias.

For everything, you guys do to help make this all a reality.

In the new year, very soon, in February, we are looking at making some changes to the podcast, all positive stuff.

Maybe I’ll finally find a way to use my neon sign.

Twenty four. Alright. Let’s start at the top.

The kind of global macroeconomic situation. I think that’s a good place to start.

In twenty twenty three, at the beginning of the year, the word among kind of that was front of mind for a lot of people when it comes to the kind of a broader macro economy was uncertainty, and for a very, very good reason. We were still kinda coming out of the pandemic and lockdowns.

I think at the time, China was still locked down. A lot of companies were uncertain how long the, supply chain disruptions would last for.

A lot of companies were uncertain of how bad inflation would get. Companies were uncertain of how what the Fed response was going to be, how long it was going to be, what the impacts were gonna be in consumer spending and employment and on and on, and that created a lot of uncertainty. And at the beginning of the year, a lot of companies kinda press pause on a lot of data related data related initiatives. I know this is to be true.

A lot of other companies, however, were remained committed and had already committed to digital transformation, so the kind of that ship was sailing. To make a long story short, there was a lot of uncertainty last year.

I think for the most part, we’re through the storm.

Now a lot of people were impacted this year.

There was some downsizing that wasn’t all kind of, you know, rosy and cheery last year.

But I think last year wasn’t nearly as bad as many thought it could be, and I think that we’ve kind of turned the corner. I think in twenty twenty four, we will start to see a bit of an economic recovery.

I think we’ll start to see, businesses spend a lot of the cash that they’ve been sitting on for a while, and, yes, a lot of businesses still have a lot of significant, retained earnings. I think there will be ongoing investments in things like AI.

Data will continue to be invested in, I think, writ large. I think a lot of companies that were holding back on investing in things like software or maybe consulting and services or large scale programs that that hit the skids in twenty twenty three will come back in twenty twenty four. So that’s a good thing.

If you are in this space, if you were working in data and analytics, maybe you’re a consultant, I think twenty twenty four is gonna be a pretty good year. I think there’s a lot to look forward to.

Yes. There’s still some dark signals out there on the economy. There’s still you know, debt levels are extremely high. Inflation remains slightly problematic.

Of course, there are wars going on, which are absolutely positively horrible. My heart goes out to people in Gaza, in Israel, in Ukraine, everywhere where there is war, and there are there’s war in a lot of places. So there’s a lot of things to be concerned about out there, but there’s a lot of reasons to be optimistic as well. And and and I think that twenty twenty four is gonna be a pretty good year when it comes to those things, and it’s going to be a good time to be in the world of data.

Now with every good comes a little bit of bad. I do honestly think that twenty twenty four will be a year where we see, at least in the private sector, where we see for the first time ever a net decline in the total number of CDOs.

I I do believe that there’s a lot of impatient and a lot of frustrated, c level executives, primarily CIOs, but including CIOs and CFOs and other c suite executives, maybe COOs, that have grown a little impatient with, the data function at their organizations.

I think a lot of CDOs who were hired in twenty twenty, twenty one, twenty two to fulfill on, let’s just loosely call them transformation initiatives, right, that were that were being hired to pivot an entire company. Tall ask, I know.

But we’re being hired to fulfill these very high level transformative initiatives within organizations, generally loosely focused around digital transformation.

Where a lot of those a lot of the CDOs haven’t been able to deliver on significant enough number of the deliverables associated to that transformation, and a lot of those CDOs, I I think, are are are turning over. That is creating a lot of interesting downstream impacts.

We’ll talk about a little bit more. Bringing a lot of actually new people into the organ into the data and analytics space because, you know, the functions haven’t gone away. It’s just these roles are turning over and the people are turning over.

But I do think in the private sector, we’ll see a net decline in the number of CDOs.

Now private sector or public sector, I should say, completely different story. So I think we’ll see a lot of those private sector CDOs turn over into, public sector CDOs when the pay makes sense.

Generally, it probably won’t because public sector, you know, you’re you’re more than likely going to be taking a significant pay cut if you are leaving a Fortune one thousand company as a legit CDO and becoming the CEO of whatever, insert government agency here.

But for many, that’ll be okay. And for money, that’ll be that’ll be, a challenge.

So I think we’ll see some of those folks take some of those government spaces or government roles, government CDO roles.

I just saw in the latest, kind of AI main mandate, they’re even talking about a a kind of a chief AI officer, which is, which which is interesting. I don’t know how they’re gonna get, qualified people to those roles, with the money that governments pay.

Spoiler alert, you’re not.

Separate issue.

So in the public sector, you’re gonna see an explosion of the number of people who are CDOs of of government functions.

And a lot of these folks are mid career. Right? A lot of these people are coming out of data and analytics functions where previously they had been maybe data stewards or data owners, or involved in in governance, type related roles, and and now they’re kind of being thrust into this fairly high level role where there’s a lot of ambiguity around what those roles actually are. Right?

What is the role of a government CDO versus the government CIO? Where where what what’s my remit? Where where do things end? How much power do I actually have?

Is this really just an administrative function where all I’m doing is defining governance policies?

I I think I think these are valid questions to be asked. I think a lot of people in these roles are are going to be asking them and need some help.

So if you’re a consultant in the government space, boy, it’s gonna be a good year for you. I’m telling you it’s gonna be a good year because there are a lot of very new CDOs in the government space who desperately need our help. And by our, I mean, that’s even includes me, right, talking about talking more about fundamentals, talking more about MDM and data quality and data governance, which is something that I’m committed to doing in twenty twenty four. So I do think we will see some attrition in the in the private sector, massive growth in the public sector, which means that the marketplace for CDO, like, CDO roles in the private sector is going to get even more competitive than it’s ever been.

My friend, Kyle Winterbottom, who runs a recruiting company over in the UK called Urbission, has once told me the number he gave me is for every one qualified you know, for one legit CDO roles role, he gets four hundred resumes, which is pretty remarkable. And if that was the case in twenty twenty three, I think you could easily consider that to be probably five or six hundred next year. So so we’ll see, but the market is gonna get really, really competitive.

However, if you if you are a CDO and have had some success in transformation centric roles, I think you could write the ticket.

Right? You could you could go from maybe a relatively small company, a two billion, three billion, four billion dollar company, and you could absolutely, positively write your ticket into a major Fortune fifty company because they need your help.

So interesting time to be a CDO, interesting time to be in the market would be maybe a good time. I think twenty twenty four would be maybe a good time to up your skills game, to learn a little bit more about AI and about how these things actually work.

I do think that a lot of those CDOs that we’re transitioning out the CDO roles, will end up landing in VP and director of analytics roles.

Again, probably taking a little bit of a pay cut, But maybe arguably going back to more of a comfort zone, more of a zone where you were focused on kind of pure data and analytics operations, right, where you are becoming more of the operator, more of the enabler of some of these core capabilities that may allow you to take a breather on your career and to get back to some of the essentials and to focus on upskilling, particularly when it comes to AI.

Because one of the other trends that I’m seeing that I that I think will continue into twenty twenty four is CDOs struggling to hold on to the AI function of their organizations. So a lot of that same frustration with the lack of delivery or lack of perceived delivery by CDOs, That holds true in the in the world of AI as well.

There’s a lot of companies out there that want movement on AI. A lot of c suites see the transformative nature of AI, which is very real.

We all see it. And they want movement, and they wanna get going. And they want to find ways to leverage value of AI. And a lot of CDOs are basically saying, hey. We’re not ready.

And this is something I’ll talk about a little bit more, and I’ve I’ve been talking about now for a while. I I just published an article. It would have been a month ago now, but I just published an article in Forbes talking about AI readiness.

And a lot of CDOs out there are saying, hey.

We’re not ready because our data house isn’t in order.

But I’ll tell you what, it hasn’t been in order for a long time.

It wasn’t in order ten years ago when you were talking about big data.

It wasn’t in order three years ago when people were getting hired to fulfill fulfill digital transformations and it’s not an order now. Does it need to get an order?

Of course, it does. Of course, it does. But that doesn’t mean that you have to wait. We can walk and chew gum. We can focus on foundations, yet at the same time, we can be innovative. We can be forward leaning. We can find ways to leverage value from from from AI now.

And those CDOs who haven’t and who have been dragging their feet and have been been maybe maybe pushing back, maybe asking for hold, maybe asking for time.

I think those CDOs are putting themselves in a risky position, and those CDOs need to absolutely positively up their game and to figure out how we can start leveraging value from AI now, pronto, as it were.

So what else is going on in twenty twenty four?

Well, one of my things about twenty twenty three was the demise of the data mesh.

I think in twenty twenty four, it will largely evaporate.

And I don’t say that with joy or with glee.

I was not ultimately pro mesh, and I wasn’t anti mesh. I had a lot of concerns with the mesh. And if you go back to previous editions of CDO Matters, if you go back to some of my posts on LinkedIn, you’ll hear those concerns. I mean, I have been consistent in this regard.

I had concerns about the complexity of Federated Computational Governments. I had concerns about the complexity complexity.

Talk for a living.

You figure I could do this.

I had concerns about the complexity of data contracts, about the the the decentralization of data management. Right? Centralization and and kind of hub architectures exist for a reason. I’ve had concerns about the data mesh since day one.

However, I also said that there were things to like. There were a lot of things to like. I happen to like and believe firmly that we will automate data governance one day.

But, ultimately, what companies found was that they lack the technical and data management and data governance maturity, for lack of a better word, that they lacked the chops as it were to fully execute on data mesh.

So what they ended up doing was embracing the one thing that they did have the maturity to execute it against. The one pillar of the data mesh and maybe arguably two pillars of the data mesh where they felt comfortable, which were the implement implementation of some form of a data product owner, because that’s easy. It’s just a new label. It’s just a new new role.

That’s one.

And some form of a data marketplace that is supposed to be the kind of the fulfillment enablement platform of of a data mesh, AKA, for most, a data catalog. So what I’m seeing out there the the the kind of the glib joke that I told in a previous edition of the podcast was was that you can if you can kind of imagine the the the data mesh as a flatboard truck, flatbed truck.

You know those trucks that that just are flat on the bottom. Like, maybe you’ll you’ll load another car on it or maybe you’ll you’ll whatever. But they’re just flat. Right?

Like, they don’t have walls and they don’t have the back.

And then all of a sudden, people were just kind of loading up everything out of the back of the flat board flatbed truck. Right? They loaded up data contracts. They loaded up federated computational governance. They they they they low they loaded up data product owners. They loaded up all the expectations and hopes and dreams of data mesh, including by the way, they probably threw a lot of dollars and bags of money into that as well by hiring consultants and others to help with the transformation towards a mesh.

But, anyway, they threw everything on the truck.

They got in the truck, and they hit the gas. Five miles later, they get out of the truck, and they look in the back, and the only thing that’s left is a data product owner. That’s kind of the joke that I’ve been telling about the data mesh. And, again, I don’t take glee in this because I think a lot of companies lost a lot of time, and they lost probably decent chunks of money as well chasing after something they weren’t ready to implement.

And we have seen this over and over and over again.

Conceptually interesting, theoretically challenging, but conceptually interesting, and I could actually see a lot of the value there. I think there’s great value in peer to peer, creating and managing some form of a peer to peer data management ecosystem at large. Doesn’t have to be just about analytics, by the way.

I think you could do this operationally.

There’s tons of opportunity for sharing data between domains, between functions. There’s lots of opportunity there, and there’s a lot of goodness when it comes to enabling individual domains to maintain autonomy and control over their analytics.

So there’s a lot of a lot of goodness here.

And, again, and I don’t take glee in this, but I think a lot of companies spent a lot of money on trying to implement something that was, in its totality, really complex and ended up having to pull the plug six to eight months later, which is the usual kind of gestation for companies to figure out that what they’ve bitten off, they cannot chew.

So data mesh will completely, largely go away, and the only thing that will remain standing is data product owners.

However, in twenty twenty four, I think you will see and this this saddens me, and I really don’t wanna say it because I don’t want it to be true, but I know it to be true.

I could be wrong. But this one saddens me.

I I think we will see significant declines in the hype related to data products.

The hype that was generated by the mesh and the requirement of a data product owner, which is, by the way, I’ve I’ve done lots of content on this.

A data product owner, the idea there is very different than a data product manager, by the way. A data product manager would be would be responsible for total life cycle manage, would be responsible for competitive intelligence and pricing and positioning and marketing and all sorts of product management goodness.

Data product owners are largely associated and responsible for some of those things, but it’s largely, I would argue, a governance function. You own.

You own policies for this field, attribute, dashboard, report, this data product. You own it.

And that’s, like, kind of the last thing standing is is is is this concept of data product ownership.

A lot of companies implemented it because I think a lot of CDOs needed to save some sort of face from money spent on a mesh. That’s the last thing standing, and I think that a lot of people started to talk about data products as a result of the hype around the mesh, and there just isn’t enough wind in those sales.

The reason why there isn’t enough wind in those sales is because most companies, sadly, have not been following a lot of the advice of myself and others like me who have been saying, hey.

There’s no value in a new label to an old job. You can’t take a data analyst and just start calling them a data product owner and have there be value. Doesn’t work that way.

The value of data products is the application of product management as a discipline to data management. I would argue there’s a ton of value there because when you take a product management and product centric approach, you put the customer at the center of everything you do. You focus intensely on good product design. You focus intensely on understanding customer needs.

You even do things like understand what customers would pay for that product. Meaning, wow. Holy cow. Wouldn’t it be amazing if we could actually know what it cost to build this stuff or manage this stuff?

Or even better yet, because I’m a product manager and what product managers are good at doing is pricing. They should be, by the way.

Not only do I know what it costs, but I know the value it can benefit. Value it can generate.

So the value is what? A coin. One side of the coin is cost. The other side of the coin is benefit. That’s the value coin.

Product managers, that’s what they do. It’s a part of what they do for a living.

So what I’ve been urging people to do for the last year is if you are interested about data products, what you should really be interested in is finding ways to hire product managers and implement product management as a discipline into data management. Nothing but goodness there. Value will be delivered because product managers, professional product managers, are in the business of delivering value. That’s what they do.

However, unfortunately, I think as a part of my prediction for twenty twenty four, data products will lose steam and will largely fall from front of mind for a lot of data leaders because they have not implemented product management as a discipline, because they have not hired experienced professional product managers to manage data products. So, unfortunately, I think that we’re gonna lose some steam there. And by data by twenty twenty five, I don’t think we’re gonna be talking about data products much at all. Sadly, that pains me.

But that’s just what I’m seeing out there, folks.

What else?

Well, let’s talk about the data fabric. If you have listened to me or a previous podcast or any of my content online, you will know I’m a believer talking about the data fabric.

And and I think that the fabric narrative as it were really matured a lot this year.

I think, you know, yes, in part to you know, Microsoft actually launched a product called Microsoft Fabric, and it’s a great v one, but it really is a foundational layer. It’s it’s not all of the things that I involve with Fabric that I that I view with Fabric will have. But I do think in twenty twenty three or twenty twenty four, we will see a lot of focus on data fabric.

But people will use different names for it, and and I think this will be one of the stories of twenty twenty four is that by the end of this year, I think we’ll start to coalesce around a few key concepts.

And and the key concepts that they involve the data fabric, which is this kind of this seamless integration layer. It’s a data virtualization layer. It’s a semantic layer. It’s an analysis layer. It’s a data management layer. It’s all of these things potentially.

Right?

That’s why I describe a data fabric as a data management architecture that can where where data informs its own classification and use because That’s a very, very loaded statement. That’s my definition of a data fabric, data management architecture, where data informs its own classification and use.

But I think what you will see is this that that that some kind of very adjacent ideas start to coalesce.

Right? So with large, the data fabric, most people see that as an integration layer or virtualization layer. Fine.

We don’t need to argue on that. Good enough. Then there’s metadata. Right?

So there’s all of these goodness, all of these nuggets all around the organization that tell us things about what people are doing, what they’re buying, what they’re clicking on, what they’re not clicking on, time series data, complex metadata, you name it.

There there are nuggets all over the place that provide a depth of insight into how the business is operating, how the business is running, right, at a very, very, very granular level.

So there’s metadata. Gartner talks about this idea of activation of metadata.

Today, we just kind of we catalog it. We we put it in a library. We put it in a dupey decimal system, and it just collects dust most of the times. We we don’t do much with it.

But if you can find a way to activate it, as Gartner would say, there’s a lot of insights there, a lot of things that you could take action on. Like, maybe even use metadata to start helping automate data modeling, maybe, or automate rules related to governance or automate how you match records in an MDM process. The list of things that you could do here is very, very long, But you have this integration layer, the data fabric. You’ve got metadata.

You’ve got things like graph and vector data vector databases that can be used to understand context in a way that simply is not doable in rows and columns.

Right? Graphs can the the triples based in graph are are basically just basic sentences.

Joe lives New York. Right? Okay. So it’s very basic sentence, but but you you get my point.

Right? So graph is sitting out there. Metadata is sitting out there. The data fabric is sitting out there.

You got AI kind of sitting out there. You’ve got all of these things, I think, that will slowly start to coalesce, where the promise of the data fabric or the promise of we don’t need to call it a data fabric.

We could call it maybe a semantic layer. Maybe a data management layer.

Maybe, as my friend Sanjeev Mohan would say, a a metadata plane.

Maybe it’s a modern data management platform. I I don’t know, But all these things are gonna start to come together, where graph and AI will be used to analyze and tell us things about metadata.

Right?

And those things that it tells us can be used by AI to start automating at the beginning, it’ll be augmentation.

Human humans decide ultimately, but machines inform. We’ll start with automation or augmentation of data management capabilities, data quality, MDM, data integration.

Data will when it comes to integration, data will tell us when things need to be integrated, when they don’t need to be integrated.

Right?

They’ll they’ll tell us when what the rules should be, what transformations may need to be. So we’ll start to see these things all come together.

AI will start to to make recommendations about ways to improve the data, not only for analytical purposes, but also for operational purposes. But what fields should be required versus not required in order to improve a business process?

So I think by the end of twenty twenty four, these things will start to come together.

What we will see, I think, and and I I I don’t know a better word than this, so so forgive me because it’s not a new word.

But it’s okay because I’m not inventing a new word because that that’s a pet peeve of mine. I I don’t like it when people invent new words.

What I what I think we’ll start to see is what may otherwise be called headless data management applications.

Okay?

So there are classic data management applications out there today, like ours, Prophancies, MDM, data quality, data integration, BI analytics, where these applications all exist today, and they manage the rules related to when is data accurate, when it’s not accurate, accurate.

How do we integrate system a to system b?

Like, that’s where all the rules live is in these data management applications. And human beings sit on top of those applications and configure all the rules.

There are administrative UIs that are core to all of these things, whether you’re building a dashboard or whether you’re defining a match strategy for MDM or whether you’re defining data quality rules in a DQ system, whatever it is.

There are humans that are interfacing through administrative UIs that start to to find those things. Those administrative UIs will increasingly become the domain, for lack of a better word, what I will see and loosely call smart agents, where smart agents will start to be integrated deeply into those applications where where they will be more recommend make recommendations.

Right? Oh, I see you’re trying to maybe build that integration or you’re building a hierarchy.

Can I can I make some recommendations for you about how entity a is related to entity b?

Because the graph is gonna tell you that.

My point being here is that over time, I see that data management will become an intelligence layer within a modern data ecosystem, and it will increasingly be more and more headless.

The metaphors here are very we start getting into decapitation and and and heads. Maybe that’s a bad one.

We need to come up with something else. But but where in the short term, it’ll be agents making rec recommendations to people about data quality rules or match strategies or hierarchical structures or integration patterns or or dashboards, where ultimately a lot and more and more and more and more the administrative capabilities of those tools will be cooked into this smart agent enabled layer that is a part of what I would loosely call the data fabric, where semantic layers and where data fabrics and integration layers and where a lot of other things just start to kinda slowly come together.

We don’t know what this feature looks like, but but I’m excited about it. I I I I don’t see MBM going away. It’s not it’s not gonna go away. I just think that machines will play an increasingly important role in making recommendations about the rules used by MBM. Same is true with data quality. Same is true with data integration. These things are not going away.

The robots are not going to take over, but they will increasingly play an important role in making recommendations.

Those recommendations are going to come from some AI enabled layer that uses metadata, that uses transfer transactional data, that uses full text data, just rows and columns, to make recommendations to us. So I think by the end of twenty twenty four, we’ll have a lot more clarity on what that looks like, but we’ll still lack full clarity because it’s an evolving space.

So that’s data fabric, kinda smart agents. Call them copilots if you want. I like that that copilots more and more copilots.

But that does dovetail into key theme twenty twenty four around AI readiness. If you go to an industry event this year, and I hope you do because they’re they’re a good investment in time.

If you go to an industry event this year, the number one thing you were going to hear over and over and over and over and over and over and over again, AI readiness. You’re particularly gonna hear it from consultant vendors because what they’re gonna ask you is, are you AI ready?

Chances are pretty good. You’re gonna say no.

Because I don’t know anybody who’s who’s who’s yep.

We’re good to go. We’re ready. We checked all the boxes. Right? We got it. We we we we got this.

Right? I don’t know anybody that’s saying that, which is which is kind of the power of these types of arguments. Right? How do you say no?

Because you don’t. It’s like, oh, you do want better data? Of course, I want better data. I really want better data.

But, anyway, it’s the number one thing you’re gonna hear at these conferences because chances are you’re not fully AI ready.

And that’s okay.

Right? And it’s okay to say you need to focus on some of your fundamentals because you do.

You you you you need good nutrition.

Right?

You can’t you can’t I we we’ve yet to live in a world where you can have your cake without spending a little time on the treadmill.

Doesn’t exist.

However, however, as I shared, hopefully, in a reasonably elegant way, in an article I recently published for Forbes, through the technology council that where I described and what I urged and what I recommended to CDOs and what I will repeat now is that, yeah, you gotta get the fundamentals right, but you also gotta go now.

You gotta move. You gotta innovate. You gotta find ways to integrate AI. Now. Met no.

Let’s go.

It’s now in French, because I’m Canadian.

Gotta find a way to integrate it now. And your c suite does not want to hear you say you need more time. Your c suite does not want to hear I need to figure out my my fundamentals because chances are pretty good. You may have said that two years ago around digital transformation.

There are lots of companies out there, and I’ve talked to them, folks.

I’ve talked to them who’ve been using AI for years and years, who have been building propensity models, who have been building look alike models, who have been building all sorts of models to predict everything, supply chain shortages, the weather for heaven’s sakes.

I mean, the list is long, and trust me, they all need help with their fundamentals.

All of them.

So two things can be true at once.

You do need to focus on your fundamentals.

However, you need to find ways to drive value right now.

Right now. Quickly. Right? Because that’s what your c suite expects. And frankly, it’s where the bar is in terms of are you gonna be able to keep pace because there are transformation impacts here, transformative impacts.

Right? If if teenagers can use Chatt GVT to do their homework, we can find ways to integrate this to our company. We can, and we can find ways to drive value now. Are there concerns about AI ethics?

Are there concerns about data quality? Yes. Yes. Yes. Yes. Yes. We can go down the list.

But this does touch on something here that I think it will be another theme we start to hear more of in twenty twenty four, which is my hope, my sincere hope in twenty twenty four is that we start to break away from old deterministic mindsets, and this is something I cover in this article in Forbes.

That we break away from these old deterministic, very binary mindsets that that say, well, I can’t do a until I do b. I can’t get value from AI until I focus on my on my basics.

I can’t do these things, and I can’t can’t get.

That is a very deterministic binary all or nothing way of thinking that just doesn’t work in a world of AI. AI exists in a world of probabilities, deterministic, a or b, yes or no, right or wrong, binary, versus probabilistic, maybe.

Right? Where where everything exists as a probability, which actually more closely aligns to how data behaves anyway.

Right? Data quality is not absolute.

Data quality is entirely subjective based on a use case, based on a company, based on the data, based on the creator, based on all sorts of different factors.

So data, data quality, the way it’s used exists more in what I would call kind of a more of a quantum paradigm, where it could be any sort of probabilities.

But to make a long story short, it’s okay to say I’m eighty percent confident this is the accurate number, or I’m seventy percent, or even to say I’m just fifty percent confident this is right.

Frankly, seventy percent of the solution is better than zero percent of the solution.

And I don’t know a lot of people who make decisions in business with one hundred percent confidence about anything.

Anything.

So if there’s one thing that I think you as the chief data officer or the data leader of your organization instead of deterministic.

Because deterministic is not how our instead of deterministic because deterministic is not how our world works.

It may be how things work in a world of rows and columns, but it is most certainly not how things work in a world of endless probabilities.

So AI readiness will be something you see.

First thing you should ask, what do you mean?

Right?

Like, what do you mean?

Right? And and and, yes, part of the message here will be, you know, to eat your greens, and that’s okay because we all need to improve our governance. We certainly do. We certainly do. But what does that actually mean to you to be more AI ready? Maybe it’s something as simple as finding a way to implement a single source of truth or a single view of your customers within your CRM database or to figure out a better rule for data quality. I mean, whatever it is.

The the the things here are very, very wrong, but of all the things, of course, that we could be doing, but but let’s be skeptical and let’s ask some questions.

Like, hey. Well, what does that actually mean? Right? Is this an eighteen month long exercise that I will spend, you know, a lot of money on and not be entirely sure what the actual outcomes are? Or can you tell me what the specific business outcome will be after we spend the next six months becoming more AI ready?

That’s the key, folks.

That is the key.

What will the outcome be?

If this is AI ready for AI ready’s sake no. We tried this. It’s called big data. Didn’t work. We were big data for big data’s sake.

It was like, maybe the train will come to the station. Maybe we’ll need this one day. No. No.

No. Hey. I already great. Okay. What does that mean? What will I be able to do tomorrow that drives meaningful business value, meaningful quantifiable business value that I cannot do today?

Done. That’s the conversation you need to have with those industry events with people who are trying to sell you becoming more AI ready.

And it’s a good conversation to have.

We need to have it. Don’t get me wrong.

You just need to make sure that that SOW, that AI ready SOW is talking about those outcomes.

You can tell I’m passionate about this stuff.

Alright. What else? AI governance. Oh, boy. Oh, boy. Oh, boy.

I could talk for hours about data go AI governance. I don’t I’ll be honest, folks. I don’t understand what the difference is between AI governance and governance.

I do know that calling AI governance different than governance is slightly potentially problematic for CDOs because that opens the door for that governance to live somewhere outside the CDO role, and we don’t want that.

Not because we’re empire builders, but because we’re data experts.

AI should live in the date in the CDO function. It should most certainly live in the CDO function. One of the trends that I’m seeing is the AI function, AI capabilities are being moved out of CDO roles and under CIOs. Yes. Even CFOs have been hearing about this, which is like we don’t want that to happen.

Again, not because we’re empire builders, but because we are the experts We need to lead when it comes to AI. This should be part of a data and analytics function.

It should be under CDO remit. That’s why it’s so important to That’s why it’s so important us for in twenty twenty four to up our game around AI.

When it comes to AI governance, yes, we need to take most certainly some sort of risk based approach, particularly if we’re in the EU. We have some mandates to do that.

But at the same time, we need to take an opportunity based approach as well. Both need to exist at the same time. What are those opportunities? Go find them and do the minimum amount of governance needed to enable some of those basic AI driven use cases.

Don’t spend the next twelve months only talking about governance.

Find some outcomes. Find some business impacts that you can make.

Find some business leaders with pains that you can help with AI.

Develop an MVP, and what is the minimum governance needed to do that one thing while you’re also taking that risk based approach.

Do both.

Don’t do either. Either is that deterministic mindset we’re talking about. Either I do this or I do that.

Either I focus on AR governance or do nothing.

Do both.

It’s more of a quantum mindset.

I like that.

It can exist. This photon can exist at two spaces the exact same time.

I like that. It’s a quantum mindset. Maybe I’ll maybe I’ll go with that. Alright. If you stuck it up this far, kudos.

Kudos. We are almost getting to the coup de grace, which is, like, my ultimate long shot of long shots when it comes to data predictions. But let’s finish off a few others.

Data sustainability gained a lot of traction in twenty twenty three. I think we’ll continue to see traction in the US in twenty twenty four around sustainability for CDOs, around data hoarding, issues related to data hoarding, maybe solving some of these issues within the US. But I think in the Europe, we’re gonna we’re gonna lose some steam.

And I think that’s just largely a function of, you know, if we’re gonna be burning coal, to augment our gas supply, I think it’s really, really difficult, for society and regulators and others to take, you know, to look, CIOs and CDOs, in the eye and tell them that they need to reduce their carbon footprint within the data centers while those very data centers are being fueled by by by coal based, energy sources. I mean, it’s just that aside, I do think sustainability will kind of hold writ large on a global basis. I don’t think we’re gonna make massive jumps forward. I don’t think it’s gonna fall completely off the map, but I do think that it’s just kind of be gonna be sitting there percolating had it because I think a lot of the changes over the last year about global energy security have kind of put put sustain a little bit sustainability a little bit on a back foot.

Data literacy. Boy, oh, boy. If you know me, not a fan. I think twenty twenty four is the day is the year when a lot of people see data literacy for what it is.

Right?

And and, honestly, folks, I I I know this sounds harsh, but for many, not all, but for many, data literacy is about blaming the skills of our users for building rotten products.

We need to improve our products.

We need to build better products.

We need to understand our customers better. We need to understand how they operate, what they do, stand behind their shoulder, watch them do their jobs.

When we do those things, when we build excellent products, when we take a design centric approach, when we take a product management centric approach, when we know our customers’ business processes better than they know them and we build amazing products and they still don’t use them, okay. Maybe we need to talk about training. Maybe we need to talk about user skills. But until we do all those things, I don’t think it’s right for us to point fingers at the skills of our users, particularly if we cannot tell them what the value of using our products is.

See previous podcasts around the problems with data literacy as well as previous article I published in Forbes titled The Problems of Data Literacy.

What else?

I think we will continue to see this is my last kinda major topic here. I think we will continue to see more and more new people flood into the space, which is good which is good. I think that’s largely a function of a lot of CDO turnover. It’s also a function of a lot of government agencies focusing on data management, data governance, all good things.

We’ll see a lot of new people continue to flood into the space. That is a good thing because our space will grow. We like that. We like growth.

Sadly, a lot of these people will come in without a ton of experience around data, analytics, governance, AI. A lot of people will come into our space, and they will look to the industry for guidance. They will look to thought leaders. They will look to pundits. They will look to professional organizations.

Maybe they’ll look to look get certified in something, certifications, you name it, professional development. They’re gonna try to build the skills they need to become excellent data leaders.

Unfortunately, what they will see and what they will experience is a lot of stuff. We haven’t been doing the exact same for twenty years.

And my hope in twenty twenty four is that twenty twenty four will begin a process where we start to recognize we really need to innovate in this space.

Our oh, our our ideas, particularly related to data governance, are old and tired and not working.

We need to revisit our programs.

We need to revisit, first and foremost, the way we think about them, the way we think about our customers, the way we think about how we provide value.

We need to change our mindset.

And I am optimistic that in twenty twenty four, we will start to ask some serious questions about mindset within the data analytics space.

Look for more from me on this in the very near future.

I’ve already published a few different pieces related to this.

I had a podcast titled Modern Data Leadership. I would invite you to check it out.

But I think we have a significant and I would argue a required need to change our mindsets in the data analytics space.

We need to think differently about some very important things, particularly the relationship with our customers, particularly how we look at data, how we look at opportunities, how we look at data quality.

Look for more twenty twenty four, but I think I think I’m optimistic. We will start to see a mindset man.

Mindset shift. I’m tongue tied today.

I don’t know why.

I don’t even my mouth is dry. Alright. Well, for the, my long shot prediction of the year. Now forgive me for the Skynet kind of doom and gloom thing here, but I believe it or not, I’ve been thinking a lot about stuff. Maybe I should think about it a little bit more or less.

But if you may recall, in twenty twenty three, with the abrupt resignation of Sam Altman and the rehiring of Sam Altman, the CEO of OpenAI, and all that kerfuffle, There were several stories that many of which I see as likely and credible around the potential of some new breakthrough at OpenAI. Now I don’t know this to be true. I don’t know. Who knows?

Nobody know. I don’t know. I’m sure there’s some people know, but, obviously, Sam Sam Alden knows. I don’t know, and I don’t have any magic insights here.

But the conclusion that I came to is is or one of them is that it’s it’s logical potentially to conclude that there was some really, really big breakthrough there and that there were concerns about the board, about how that breakthrough was was being handled.

Project within OpenAI that was that I believe many are calling QSTAR, which which is a a a a name related to Q learning and STAR algorithms, it’s this combination of of of of a new form of algorithm, or maybe it’s not that new, that is very good at complex problem solving, right, where it very quickly finds its way, develops a path from a starting point to an end goal to the problem being solved. Mathematical problems are probably a good way to look at this. Is it do I go here? No.

No. Probability high? Yes. Go here. Probability low low higher higher, where you could work your way through a very, very complex vector space to get to the end goal of actually problem being solved.

Well, I happen to think that sometime in the future, potentially even twenty twenty four, where somebody, and maybe it’s OpenAI, maybe it’s some Google, maybe it’s DeepMind. Who knows? I I don’t know.

But where we start to get to the point where we have artificially intelligent algorithms that are one hundred percent accurate in mathematics. Where today, LLMs I think it depends.

I mean, this is one of the how they test the intelligence. I mean, they’re generally testing around seventh, eighth, ninth grade, I think.

But where in the future, it is like a hundred percent.

Like, where you’re talking like super brain figured out all the math, all the all the existing math problems and maybe even solving novel problems.

That’d be more AGI, separate issue.

But I actually do see a world where if you start doing that in a quantum environment with compute capacity that is beyond head and shoulders beyond where we are today, it is entirely possible that all methods of modern encryption could be broken, well, like, easily.

I I don’t say that’s easy today.

I I’m saying that in the future, maybe even late twenty twenty four, where we’re having very difficult conversations about technologies that can break encryption.

And if that’s true, oh, boy.

Just about everything we do, at least online, in our modern world, depends on encryption.

Depends on it. Whether we’re talking about sending money, even sending an email. Even what I’m doing right now is being partially encrypted online, I’m quite certain.

So, I mean, yeah.

Yeah.

If if AI can be used to solve very, very complex mathematical problems, and it can do it in a in a quantum computing environment where it is conceivable that some of the what we have viewed as the unsolvable mathematical problems are now actually solvable.

What does that mean? I don’t know.

I think it will call for very interesting times.

That’s for sure.

So that’s been my doom and gloom, but that’s kind of like the long shot. Is is that is that AI gets to the point by the end of twenty twenty four where it’s able to solve extremely complex math problems, and probably the only thing holding it back from breaking encryption is a lack of GPU capacity to do it. But I think in theory I think in theory, capacity will exist. In practice, it won’t because I think you’re you you need to throw a ton of compute at this thing. You would need to throw a lot of compute at it, but I think in theory, it could exist by the end of twenty twenty four, which will have some very, very significant impacts and some very interesting questions that we’ll need to ask answer as an industry.

So with that, that’s twenty twenty four in a nutshell.

Lots of interesting things to come.

It’s a great time to be in data and analytics. It’s a great time to see be to be a CDO. There’s so much opportunity here, folks. Our organizations desperately need us to lead.

They need us to drive value. They need us to figure out AI. They need us to get the foundations right, but they need us to help extract the the value from data because it is there. We know it’s there.

I am excited about what’s to come in twenty twenty four. I hope you continue to tune in to the podcast, that you continue to support me on LinkedIn, that you continue to check out what we’re putting out on YouTube, continue to check out the white papers that we’re publishing, on prophecy dot com. I think I may actually even be able to get a book done in, in twenty twenty four, maybe? Question mark.

But to anybody who supports me, my content, thank you. To my company. Thank you to everybody who support me in what I do at Profisee. Thank you.

It’s gonna be a great year. Thanks for tuning in. We will see you on another episode of CDO Matters some time very soon. Bye for now.

ABOUT THE SHOW

How can today’s Chief Data Officers help their organizations become more data-driven? Join former Gartner analyst Malcolm Hawker as he interviews thought leaders on all things data management – ranging from data fabrics to blockchain and more — and learns why they matter to today’s CDOs. If you want to dig deep into the CDO Matters that are top-of-mind for today’s modern data leaders, this show is for you.

Malcolm Hawker

Malcolm Hawker is an experienced thought leader in data management and governance and has consulted on thousands of software implementations in his years as a Gartner analyst, architect at Dun & Bradstreet and more. Now as an evangelist for helping companies become truly data-driven, he’s here to help CDOs understand how data can be a competitive advantage.
Facebook
Twitter
LinkedIn

LET'S DO THIS!

Complete the form below to request your spot at Profisee’s happy hour and dinner at Il Mulino in the Swan Hotel on Tuesday, March 21 at 6:30pm.

REGISTER BELOW

MDM vs. MDS graphic
The Profisee website uses cookies to help ensure you have the best experience possible.  Learn more