Pinar Ozcan: All right. It seems like our numbers are stabilising a little bit. So let me welcome everyone to this second webinar of the Oxford Future of Finance and Technology Initiative. And we are very excited to be talking about data driven innovation in finance. And I'll introduce our speakers in a second, but just to give you a background of what we're doing and why we're doing this, this is a way for us to showcase the research findings from the Oxford Future of Finance and Technology Initiative. We do research in this initiative through a wonderful set of researchers and two of them you will see today on open banking, AI in finance and the role of AI in the changing industries in general, as well as the importance of financial wellness. Pinar Ozcan: Our first webinar, which was last month, was on open banking. The second one today is on data-driven innovation in finance, perhaps the largest topic that we will tackle in this series. And the third one is going to be on financial wellness, which is going to be 4th of July together with guest speakers, one financial wellness entrepreneur from the Middle East, as well as Janine Hirt who's the CEO of Innovate Finance. But without further ado, let me introduce everyone who you will hear from today. We will first hear from Jack Fraser. Jack is a brilliant postdoctoral researcher in technology and entrepreneurship. He has been working with us on two years. We're going on two years, looking at the use of AI by startups and the kinds of tensions that, that creates when the technology is about AI and machine learning. So really exciting and interesting stuff that is relevant, not just for entrepreneurs, but also for investors and for institutions, such as large banks for example, that might be looking into working with AI FinTechs. Pinar Ozcan: And we will also be hearing from Dize Dinckol. And she's also postdoctoral research fellow. And her research is within the area of FinTech innovation. She's looking into how FinTech startups looking to creating value added propositions within finance, looking into particularly their data access issues and trying to understand and provide a realistic view of what it is to be a FinTech these days. And very excited to have Steve with us today. Steve is the global head of innovation at HSBC, and we also work very closely with Steve because we have just launched a FinTech programme for HSBC, and we developed some amazing content for them, which was great fun to work on. And Steve also thinks a lot about AI and blockchain and quantum and technologies in the world of finance. And it will be a pleasure for us to share these results with him, get his feedback, as well as ask him some questions and maybe some questions that he might not want to hear. Pinar Ozcan: So welcome all, it's a great pleasure to have you with us. And without further ado, let me just open up by having a quick word with Steve, to talk about data driven innovation in finance, and maybe as a way to think about it, I'll propose, how does it differ, in your opinion, Steve, to the way that the financial industry innovated in the past? Steve Suarez: Thank you Pinar, and hopefully you guys can hear me. I'm actually here in Amsterdam at the Money 20/20 Conference. I'm trying to find a quiet place so that I can get across what's going on. So, if anything gets cut off, just let me know. Pinar Ozcan: We hear you beautifully. Thank you. Steve Suarez: Good. So innovation really starts with creativity and curiosity to solve problems. And what we want to do is make sure that we're solving the right problems and we want to solve, find the job that needs to be done. And once we do that, data driven innovation is key. Without the right data, you're just guessing at solutions and that's a waste of time and money. And obviously with all the type budgets and constraints we have, we want to make sure we're using our resources effectively. So data driven innovation and financial services enables us to be more proactive and our customers benefit from getting more personalised services. So I'll give you a couple examples, like faster loan approvals, faster time to money. Another benefit of having a lot of data is that we can reduce bias in some of our decisions by having that data. I like using analogies and if you don't have the data, it's like taking journey without directions or a GPS, you might end up at the right place, but you have a much greater chance of success with the tools and the right information. Pinar Ozcan: Fantastic. Thank you, Steven. In fact, data driven innovation is important maybe also based on the research that you will hear about today. For us it's important to think about innovation in finance being data driven, because part of what it does is using data to understand behaviours in the past allows us to not just predict and therefore make products better, faster, cheaper by targeting the individuals that need them, and having more personalised products for them, et cetera, but also allows organisations who analyse the data to then nudge behaviour and make people better with their finances and make people understand and behave in a way that is more financially responsible, which is one of the biggest issues, both in developing and developed parts of the world. And so the importance for data-driven innovation for financial wellness is huge. And in fact, we see this both in the innovation that comes to the university through FinTechs, but also I'm sure that you see this in your circles as well, Steve. Absolutely. Great. Steve Suarez: Yeah. Pinar Ozcan: Thank you. So without further ado again, part of our promise to you, our audience for these webinars is that we don't go over an hour and we take only an hour of your time and hopefully make it worthwhile. Let me move on to our first set of research findings. And Jack is the main researcher here who we're going to highlight. Jack welcome, and it's a pleasure to have you in this programme, and it's been a pleasure to work with you in general. What will you tell us today about data and data driven innovation and machine learning and startups? Jack Fraser: Great. Thank you so much for that introduction Pinar. So the research we've been working on looks at the role of training data in shaping how the firms that commercialised machine learning technologies developed. So the central idea here is we're looking at how pathways are shaped by training data and how that's different to the pathways that may exist in other forms of digital entrepreneurship. So, I think to start, it helps to zoom out for a moment to just have a quick look at the components of what makes up a machine learning capability. It really involves two key central elements. The first is, it's a machine learning algorithm. And these can be developed in-house, quite often they're also acquirable on platforms like GitHub. These are quite accessible and can modify it to different ends. But the central part of this technology is the combination of the algorithm with training data. Jack Fraser: And that combination allows the development of a trained model. And this model is able to recognise patterns in data. So, that can be through things like cluster analysis, it could derive predictions through regression, and it can identify distinct categories in data using classification analysis of different kinds. So, this trained model is the core of a machine learning capability. And with these components in place, the training model is then able to process new data it encounters. So input data that comes in once the model has been trained. And that allows this new input data merge with the trained model allows the capability to develop predictions about what's likely to happen based on the input data coming in. And then from that, it's quite a short step to making decisions as well in the best interest. Jack Fraser: So a good example here, I think, is if we think about collision prevention systems in semi autonomous vehicles, those are machine learning models that can predict the likelihood of a collision. It can take new input data about things in the environment, like an obstacle that's appeared in the road and then make a prediction about the likelihood of an incidence, and then a quick decision about what to do. So it can, for example, put the brakes on faster than a human driver could. So that decision elements is central to why machine learning can be so valuable for organisations. Jack Fraser: So, this distinct structure of machine learning technologies led to a hunch amongst us, that the process of commercialising machine learning technologies might be different to that of other kinds of digital technologies for the different structure. So broadly speaking, digital entrepreneurship understands digital technologies as quite editable. They can be reprogrammed, they can be adapted and changed as the context requires. So it's possible to release new versions of software, for example, as needed. And that gives companies that develop that software the ability to adapt quite rapidly as they need to. But in the case of machine learning where the technology is really very strongly shaped on the data that you put into, it's not quite clear that, that's the case and there might be different pathways. Jack Fraser: So to explore this, we conducted a comparative study using an accelerator called the creative disruption lab, who they have sites all over the world, including one here at the Said Business School in Oxford. And so CDL is an accelerator for tech ventures. And we tracked through eight years of archival data, how machine learning ventures travel through this early entrepreneurial phase, the phase in which they're engaging with accelerators to get an understanding of how data shapes the pathways they can take. And what we found is this multi phase process where in the early phases ventures that are developing machine learning capabilities are often quite keen to pursue options in multiple verticals. But they're encouraged from a very early stage by investors to be very, very focused and develop a clear use case in a single sector. So this is a common beachhead strategy that's used in digital entrepreneurship. Jack Fraser: And at a certain point, either because of concerns about competition, or because they had begun to saturate the initial sector they were in, they were encouraged to then go back and explore applications in other verticals. What we found is that once they'd developed a very clear, well defined predictive capability in one sector, it actually became quite hard for them to move out. And the reason for this is that they've created a very highly specialised trained model that couldn't be easily applied to other sectors, trying to apply it to these other markets reduced the predictive capability. So what that really means is that early decisions about data can shape the pathways that are available to the venture. So the kinds of data that is involved in the training process can shape what they can do afterwards. And that's data that can't easily be swapped out and substitute to the latest stage. You would have to retrain the model from the beginning. Jack Fraser: So one example from our analysis was a venture that we're calling ScriptX for the purpose of anonymity. And ScriptX had developed a capability, they [inaudible 00:13:26] developed a predictive tool for writing letters and correspondence to all kinds of stakeholders. So initially there was interest from both the public and private sector from any kinds of organisations that were dealing with large groups of stakeholders, but they had this initial early traction within the public sector with governments, both with political office and with government departments. And that led to the model being trained using, first of all, public sector correspondence, which gave it the correct sentiment and tone for writing letters and that, but it also drew on a siloed databases of legislation to fill in these letters with content about which legislation had moved where. And what they found is similar to all of the case now study. Jack Fraser: When they tried to move back into the private sector, having trained their model, that was a real struggle, and it actually would've required completely retraining their model again. What was interesting with this case though, was this because the government sector had users with quite diverse and multifaceted needs, they were able to find new applications within that sector. So what originally started as a tool for writing letters became used by exactly the same individuals, which is mostly government researchers, as a way of generating reports about legislation quickly. So they were able to experiment with the tool within the sector, even if it wasn't possible to move it out across into other applications easily. Jack Fraser: So there's a few implications we take away from this, but I think one that I'd like to leave you with is, we've been through Zeitgeist of the last 20 years, defined largely by Facebook's move fast and break things. The idea that it's important in digital strategy to get early to the market, to live AB test safe in the knowledge that you can edit and change things as the market needs later. But from our research, we suggest that this might not be the case with machine learning technologies, which might have something a little bit more in common with physical hard technologies instead. So it will require a different kind of strategizing and a much more conscious choice about which kinds of data you use in the early phases of developing. Pinar Ozcan: Fantastic. Thank you so much, Jack, for this overview of our findings. And in fact, this study really has lots of insights for entrepreneurship and for being able to really see what machine learning does as a technology to a startup, and also has implications for, of course, the investors of these startups, as well as the partners of these startups. And someone who is dealing with these issues in his role is, of course, Steve, and I would love to turn to him to get a few reflections both in terms of the study itself, but also what this means significance for HSBC. Steve Suarez: Thank you, Pinar, and thank you, Jack. Actually, I found a lot of that incredibly interesting. When you think about large organisations like HSBC, we have a really broad spectrum of experience in AI, and it's important to know where to use AI, or where you can use AI, and where not to use it and where it doesn't make sense. We find it in areas around things like advanced analytics, chatbots to help shortcut information for sourcing information. And then in our sophisticated model techniques. It's very important for us, and I really want to make sure that we point this out, is that AI isn't there to replace humans, it's there to augment and help them with their abilities. So that's really key. From a challenges perspective, we're a regulated organisation. And so we have a commitment to our customers, commitment to our colleagues and the regulators. Steve Suarez: And therefore we've created a data and AI principles that include things like, the ethics, the transparency, the explainability use of AI. And it doesn't matter for us. So in our organisation, it doesn't matter how accurate the AI comes back with results. If it's missing any of the stuff that I just mentioned, we are not allowed to use it. We can't use it. And so we need to make sure that we can show how we come to our decisions to ensure that there's not any underlying bias in that. So the other thing that we're looking at is making data available so that we can build the models. And so we're exploring techniques, and I don't know, Jack, maybe you can talk about the use of synthetic data. For us it helps us accelerate innovation. One of the things that we needed when working in FinTech is giving them the access to the data so that they can show us and they can prove the value that they can provide. Steve Suarez: And we've also created internally for our employees to help them work with their AI models and their data, is a data marketplace where we can give them data sets that's in a box and then they can get going, and then they can reuse these data sets. And then from our perspective, we give them safe environments and sandboxes. So we give them a sandbox so that if anything happens or whatever, it's limited to that, it's like a bunker and you can't really create any damage and we can look at these experiments. Obviously being part of an innovation organisation, we're constantly experimenting. And then from an upskilling our people, I guess we're looking at things like giving dedicated training paths to make sure that not only we're truly understanding the technology that we're playing with and not just giving people the access to the technology and not understanding the concepts, or what's getting needs behind it. So, Pinar, I think from our perspective and tagging into what Jack mentioned, I think that, that's the way we're looking at it as well, but I did find his findings very interesting. Jack Fraser: I think just quickly on that note, I think synthetic data and data marketplaces will be an interesting part of this puzzle for machine learning, entrepreneurship, having access to those kinds of resources will be hugely valuable in the cases of the ventures we were looking at, because these were early stage startups. The issue of retraining your model, it was often both capital and time intensive, prohibitively intensive, and being able to have access to those kinds of resources would change the dynamic of particularly how capital intensive that would be. It still takes time to do this, but it would now have a pathway that has more in common with traditional digital entrepreneurship, where you can take the time to reprogram things. So I think we would see a different dynamic with access to those kinds of resources. Steve Suarez: Exactly. Pinar Ozcan: Steve, maybe just to follow-up. I think that you're highlighting the difficulties and intricacies of using AI correctly. And there can be many alarm bells around AI if not used correctly. As we know the data that we deal with in many cases is biassed to start with, how do you unbias it? How do you make sure you don't generate new biases? How explainable is it? These are all critical questions. And I'm glad to hear too about the rules that you have in place in-house. One question when it comes to combining Jack's research with some of the ideas that you've put forward is how does, obviously AI really matters for an organisation like HSBC, we know that, but it's also very intricate. How does that affect your ability to work with FinTechs on AI? Steve Suarez: I think we can, we got to make sure that it can't be a black box. So when we make decisions, we can't just say we've made these decisions. We don't know how it came out. So we need to make sure that we understand the underlying technology. The bank is an organisation of trust. And so if having that trust is paramount and anything that we implement, again, has to be something that we make sure that it's in our customer's best interest and we have that. And so I think we want to use AI, we want to use AI smartly, but like you said, it could be a tool of mass destruction that you can implement it and you can implement something really at scale very quickly and badly, and it goes badly and it does it very effectively back. And so that's why we want to be very cautious, but it's a fantastic tool. And I think every organisation is using it to its benefit and we're no exception, but I think there is a matter of making sure that you have the right skills, the right people, and you're applying it correctly. Steve Suarez: And that's why our policies and we're very strict. We actually have a steering committee that meets based on the ethics, the transparency and what gets used. So I'm actually pretty proud to hear that we're doing it, but sometimes you feel like that might... In those type of mistakes. Somebody used the analogy, having these policies, it's like brakes. And I think it's a good analogy because I don't think if you guys were driving a performance sports car, you wouldn't be driving it without brakes. And I would say that the better the brakes, the faster you can drive, because the brakes will help you slow down or stop when you need to stop. It's there for our protection, but I think AI is going to be in our future and we're definitely very optimistic about how we can use the technology to serve our customers better. Pinar Ozcan: Fantastic. Thank you so much, Steve. And now in the interest of time, let's move on to our second conversation, which is around opportunities and challenges of data-driven FinTechs. And here we have Dize Dinckol, who's our other brilliant postdoctoral fellow. The slides are moving on without me. Dize has been working on FinTech since her doctoral degree and is now a postdoctoral fellow at Oxford working with the Centre for Corporate Reputation and looking at the role of trust in AI startups particularly in FinTech. So Dize, take it away. Dize Dinckol: Thank you, Pinar. And hello everyone. So I will talk about the value that data-driven FinTech bring in financial services, as well as some of the challenges and opportunities that they face. And I'll also talk about some use cases during that. We can go to... Yeah, thank you, Pinar. The main thing that these financial services, startups and firms that do is basically bring innovation by using new kinds of data, as well as new approaches to data analysis. So one example that we see that difference is affordability assessments. In the traditional approach in order to build a credit history, you would need to use credit cards, for example. However, in order to assess affordability, these financial technology startups and innovative services are using new kinds of data, such as for example, rent payments so that they can assess how credit worthy you are, which is a different approach from the traditional approach. And also firms like Affirm and Klarna are basically using AI in order to instantly approve loans at the point of sale, which is also a quick and different approach to loan provision. Next one, please. Dize Dinckol: Another way to use this new approaches and new data is to improve security as well as enable faster decision making. We see that from established players, such as big banks like HSBC and Barclays, for example, we see that they're utilising biometrics data to improve security and basically make it quicker to verify someone's ID. Another example is a FinTech called Feedzai, which uses AI in order to detect fraudulent card activity, which also improves security. And what we see from these FinTechs are also they're using the right kind of data for their specific purposes, for their specific services, as well as utilising new delivery channels in order to lower costs. An example is salary finance, which is a FinTech providing loans going through the employers' HR systems. So they can access income data through the employer and directly cut through payments from people's salaries, which reduce those risk and enables people to receive loans through lower cost. Dize Dinckol: Another example is what we call personal finance management apps, which are basically robot assistance providing affordable financial advice to the people who need it the most. These ones basically encourage better money management practises by providing nudges gamification so they can introduce savings challenges for example, which encourages you to save more. And they also provide personalised recommendations, which enable people to access the right services for their specific needs, which also improves efficiency and customer satisfaction. Overall, all of these services improve financial inclusion of people who might have been historically basically left out of the traditional financial services systems and address unserved customer segments, which also improves financial balance of customers. However, in providing these data driven services, FinTechs also face some important challenges. The first one is around data access as well as data transportability. And we see these challenges across firms within financial services, across industries, as well as across geographies. For example, it's difficult to access data for a FinTech, which their customer might have in an established bank, for example. Dize Dinckol: Open banking regulations are trying to address this challenge. However, it's still might be difficult because of lack of standard APIs, or lack of working APIs. Another problem is accessing data from firms in other industries, which open finance is emerging to address as well. But the challenge across geographies is still [unresult 00:29:24]. This is a case, for example, for a FinTech that started in the UK, it might be very difficult for them to access customer data from a different country, for example, in France, or this is the case even across countries within the EU, which has the same regulatory framework. It is also a problem for individuals, for example, if I'm moving to the UK from another country, I leave behind all of my credit history and financial data, and I have to start from scratch. Dize Dinckol: We also see from the point of view of the FinTechs, there is an important decision to make in going to the customers through a B2B channel or through a B2C channel. What we see in the B2C area is that these FinTechs really struggle to scale because trust is an important obstacle for them to overcome. And because they might not have the trust of the customers and they don't have the brand recognition, it's difficult for them to acquire customers, which means it's very difficult for them to access data. However, they need data in order to provide data driven services. So we see them basically pivot to B2B and start working with large institutions with the data and with the customers trust such as big banks. Dize Dinckol: And the last point that I want to make is it might also be a challenge to strike the right balance between data analysis and privacy, because it's important to be able to convince the customer that what they do with people's data is relevant and is valuable to them and being transparent with how they use the data, how firms use the data, how firms collect the data, what data they collect, why it's important to collect that data. It's important to tell the customer those points and also convince them to what they get from that data analysis will be valuable for them. So it's important for financial services firms to basically have a privacy first mindset as well. Looking forward to hear what you think, Steve. Steve Suarez: Yeah, no, I think you're spot on. It was interesting. And when you were talking about the authentication, and I'll go back to some of this stuff we were talking with Jack earlier about the AI stuff, and that's come a long way. I'm at a conference and there was a company, we actually invested in this company called Callsign, where they gave me the guy's username and password. It's like, here's my username, here's my password. Go ahead and get into the system. And they take so many different, they look at how you type, they look at the machine, there's so many different things that they do, and they're able to de to detect it wasn't that person logging in, it was somebody else that compromised their username and password. So I agree with you. Trust is critical. And for us, it's one of the most critical things. And so we prioritise our security over everything. Steve Suarez: So that's when we do type of testing, we make sure that we do it with public or mass data. And I'll go back to the synthetic data is where, we have data that we want to use, but we can't use that data because of data sharing policies and stuff like that. And we have the ability to look at some of our data and create data, or agent based data, to mimic some of the same behaviours so we can get the value out of the data without compromising some of our data sharing opportunities. And for us at the bank, we're working with organisations like, I'll give you an example, like The Alan Turing Institute to really understand some of the work that they're doing so that we can actually work from a data perspective. When it goes to FinTechs, and some of the best practises to be working with FinTechs, and this is something we remind our people all the time. They're very small organisations, and they have small budgets. Steve Suarez: And so sometimes the amount of rigour and requirements we put on them could put them easily out of business. So they're some factors that we want to make sure that when we work with FinTechs we're clear on. So, one is making sure we have clear and transparent communications back and forth between us and the FinTechs. It's important to make sure you have very good KPIs that define what success looks like. So at the end of it, it's clear, did I succeed? Did I not succeed? And to give them feedback. And before I move on to the last one, that's key, because I think as a big organisation and I've seen some FinTechs that we've turned into valuations that have just done amazing is that they listen to some of the feedback that we give them and they edit their product or they enhance their product. And because they're working with a bank like ours, once we create a really good product, then they can go and commercialise it with other partners because we put in the standards and the structure in place to make that successful as a FinTech. Steve Suarez: And I've seen some really interesting FinTechs flourish under that type of direction, actually, it's made me really, really excited to see how well they're doing. And I think the last part is, we provide them, and I mentioned it earlier when I was talking about Jack, so we provide them sandboxes so that we can experiment and have clear visibility in what's going on back and forth between the organisations. But I would say, yeah, your research is pretty spot on in the stuff that we see here at the organisation and at the bank. Pinar Ozcan: Thank you so much, Steve. This is wonderful. Jack, over to you. Jack Fraser: Thank you, Pinar. Steve, I actually have a question that just, I think builds off, off the back of the last point. So actually one of the questions in the comments from Diane Kane was about the use of regulatory sandboxes. So the UK utilises regulatory sandboxes for FinTechs, what has been the impact of this? And that's something I'm curious about as well about specifically the regulatory side of that and how that works both for FinTechs and for large financial players as well. Steve Suarez: I think it's fantastic because then you can try things out with the regulators oversight and you can see and make decisions a lot faster. And obviously we're in 64 jurisdictions. And so we have to work with a lot of regulators all around the world. And so we've been doing a lot here in the UK, even in Asia I know we do a lot of work with the MAS and using those sandboxes and testing some of that stuff out. I was just trying to reflect, am I allowed to say something, but soon you'll be hearing about an award we won based on this type of activity that we are working with a regulator using their sandbox, and we were able to prove some very valuable stuff out. And we even get innovation credits to where some governments actually says, if you do these [inaudible 00:37:03] with us, we'll give you tax credits and stuff like that. Steve Suarez: So there's plenty of opportunities for large corporates like ourselves, but also FinTechs to work with these regulators, to see how we can solve some of the problems that they're trying to solve as for the whole industry in their markets. Hope that answers that question. Pinar Ozcan: Thank you to the impact. It was great to see what you're doing within the sandbox space. And I think, Dize, you have also seen different types of sandboxes, data sandboxes versus regulatory sandboxes. Do you want to tell us a little bit what you have seen from your research? Dize Dinckol: Exactly. So the regulatory sandbox in the UK that was offered earlier in the earlier days of open banking was mostly about testing almost finished products with real customers, and didn't offer any data for FinTech services to train their AI and basically try to improve their algorithms. But what we see from that regulatory sandbox experience was mostly proving the success potential of bank FinTech partnerships. And that's when we start seeing some of the pivots from B2C to B2B in these FinTechs. And I know that there are also now initiatives in terms of providing a data sandbox from the UK government as well. I'm not sure if it's life, but I know there are works towards that. Pinar Ozcan: An impact. And that is a critical initiative for FinTech to survive and flourish, because as Dize pointed out, this chicken and egg issue, they get of needing to build algorithms and not getting access to data is one of the biggest initial hurdles that FinTech faces. And, of course, Jack research also points to how important it is. And in fact, when they get traction with an early customer, and especially a large customer with lots of data, they get very excited naturally because the data access is such an important problem for them. But then sometimes we call this the curse of the first customer, because that customer really puts them down a certain path of building an algorithm for a very specific use case and entrepreneurs sometimes look back and say, is this the company that I wanted to build? And that is an interesting pathway for them to see. Pinar Ozcan: So with that, as you've seen, we have swiftly moved into the Q&A section. And there's one question there, which I'm going to... I think it's for Steve and Jack potentially, it's from Tony [Fajimolo 00:39:44]. And the question is slightly controversial, just to warn you, he's questioning your comment about AI not replacing people, and particularly thinking about AI chatbots to help deliver customer service. And just wanted to hear your views about where AI replaces people and where not. And I think that Jack also thinks about these issues quite a bit. Steve Suarez: Yeah. Maybe Jack, do you want to start or you want me to start? Jack Fraser: Steve, I think you go ahead. Steve Suarez: You're always going to have, to have a human in the middle at some point you can't, if the regulator says, look, you've made the decision, who do you hold accountable? You can't hold accountable an AI bot, you can't say, okay, we're going to hold this accountable, say AI bot. So there's got to be accountability. It's not something that you can just blame the machine and say, okay, it's the machine's fault. As an organisation, people have to make those decisions. So yes, there are certain tasks that will be removed through AI. So I'll give you an example, data scientists, maybe I'll ask a quick question to you guys. How much time do you think the average data scientist spends on data cleansing and data wrangling? What do you think? Anybody want to post it? I don't know if I can see the chats on here, if we have the chat, just throw out a number percentage wise, start thinking about what do you think people are... How much time does a data scientist? Steve Suarez: So this is a person that's gone to university, gotten their undergrad, got their MBA, got their PhD, focused on, we want to use this person's brain. We want to use their expertise. What percentage of their time do you think that the average person spends on data wrangling? Anybody want to venture to guess? Pinar Ozcan: I'm going to go with 50, Steve. Steve Suarez: Yeah. Well, it's over 50, with 99% yeah. So it's over 50, it's good 60, 70, so think about, you went to university and you're spending your time just cleaning data and wrangling data. And so here's a great opportunity where you could use AI and machine learning to actually do some of them mundane work and get that out of the way so that we can use your expertise and your intelligence to make sure that we're managing that data. Now, are we removing, or are we eliminating people? Yes. I'm eliminating tasks that you have, but I still need that data scientist to actually do the work that we originally needed to do. Steve Suarez: So, if we can remove the mundane part of people's jobs so that we can use them on higher level type of activities, that only brings better surveys, we can do a lot more with that. And so if your job is only data wrangling and data cleansing, then yeah, you're absolutely right. Yeah. If that's all you're doing, then possibly AI could do that. But I think, when I was giving that example, that's where I was coming from. I don't know Jack, if you want to share any other examples. Jack Fraser: Yeah, that definitely makes sense. I think there's a few situations in which a human isn't necessarily, doesn't need to be those part of the loop. I think those are situations where speed of decision making is very important and that can be done faster by a machine than a human and where the level of prediction is higher than a human could make through just natural judgement . One thing I would add to that, there's a very interesting machine learning case study from about five years ago called the Camelyon grand challenge, which was a challenge in the machine learning for healthcare sector to get people to develop tools that would help with oncologists or with cancer diagnosis. And interestingly they found that while the algorithms were less successful than humans, algorithms and humans combined were far more successful than humans alone. Jack Fraser: And the reason for that, of course, is that machines are better different kinds of judgments to people. So algorithms are very good at making quick decisions at scale in the way that people can't, but physicians are much more able to explore fringe cases to reinterpret data under different lights and those more nuanced areas they were able to improve on. So there was a natural division of labour that emerged there, which I think is interesting in the FinTech space as well. Pinar Ozcan: Thank you, Jack and Steve. And in fact, one of the things that I want to pick up for this question is that while, in fact, focusing on what Steve has said, the nature of what the humans do is changing and will change, and that does put a responsibility on educational systems around the world to train humans in the right way for them to understand machine learning, and for them to be able to be that human in the loop in order to make the decisions, understanding what the machine is giving them, because as you know, machines give you what you put into them. And so that means explainable AI, that means unbiased data. And so we have quite a ways to go to be able to establish this globally. And I just want to recognise that we, as Oxford are very much thinking about this and making sure that our students are being trained in the right way. We, of course, have ways to go as well in that department. Dize, do you want to take the next question? Dize Dinckol: Yes. So another question to the panel, and specifically to Steve is, how well placed are big banks and organisations compared to FinTechs in terms of providing the right products and solutions to their customers, given FinTechs are more agile and flexible and all than big organisations can have legacy systems and silo infrastructures and hierarchies. So what could big firms to adapt to this fast paced revolution? Steve Suarez: I'd like to say, I think the AI is biassed because it's sending all the questions to me. And so, no, I'm just kidding. You know what? It's not mutually exclusive. It's not like big organisations or FinTechs. And I think Pinar knows this really well. The key to the future is not that we work opposite or in conflict with each other, the key to succeed is the combining of the FinTechs, the very bright people out there doing new solutions, more innovative stuff, approaching our old problems in new ways, but with the resources of an organisation like ours. And so what they lack we have, they lack the data. We have the data. They have some ideas on how they can approach it, we lack sometimes that ability to look at the same problem a bit differently. And so part of it is, it's really I see it as a fantastic opportunity in front of us where we can work with these FinTechs and they can work with some of our problems and identify it. Steve Suarez: So I don't think it's mutually exclusive, and it's not a competition between us and them. Hopefully, you may not agree, but that's the way actually, after I took Pinar's class, that changed my mind on how I look at FinTechs. I don't look at them anymore as my competitors, I look at them, it's like, how do I partner up with these people and how do we deliver something? And as soon as I call, they get all excited saying, oh, we're going to work with HSBC. And I love to prove value. And if these guys can deliver value, man, we can do amazing stuff. The bank, we process 1.5 trillion, that's a T, transactions a day. So can you imagine if I can have an organisation that can help us do that more effectively, more efficiently, you can have an enormous impact. So, anyway, so long answer is I see that we can work together. It's not one or the other. Pinar Ozcan: Thank you, Steve. And in fact, maybe if I may just add a little bit, and Dize feel free to also add from your research, there's a clear win-win between FinTech and large financial institutions. The issue is whether they're able to effectively work together. And I want to highlight maybe a couple of factors there always between an innovative startup and a large incumbent, there's always the issue of cultural fit. Are they speaking the same language? One party shows up in ripped jeans, the other party's in suits. This is the classical stuff that we teach in management schools, but there's also an underlying and potentially a deeper issue of, is the incumbents data ready to be analysed and to be looked at? And so that brings questions of data harmonisation and a deeper understanding and standardisation of the data in order for FinTechs to have a chance to be plugged in. And just want to emphasise that it's not, when we think about these collaborations, people always think about culture, but there's also IT that we need to think about. Not sure whether you have any reactions to that, Steve. Steve Suarez: Completely, completely, and that it is sometimes, I actually enjoy it myself, because I sometimes working with some of these FinTechs, I actually absorb their energy, their enthusiasm, and I get it really excited. So, I'm very lucky in the position I'm in, but it is hard because sometimes they're like, well, why don't you do it this way? Well, we're a bank. We need to make sure we have these protections of, well, why don't you open the whole internet so that everybody on the... You can give access to everything. Again, we're a bank. I can't just give that access. So it does get frustrating, but there's ways of doing it. And I think that's why there's innovation groups like my team that are given a bit of leeway to say, all right, we're experimenting, we're doing some of these things, again, we don't want to lose the trust and we make sure we put everything in place to have strong rigour so that we don't have that. But yes, I see that all the time. Dize Dinckol: Now actually from that point of view, one of the things that we hear a lot from the FinTechs is how long it takes to work with a bank. So I think as you said, it's important for the bank to make the rigorous assessment of whether this FinTech is trustworthy, whether their systems are robust enough, et cetera, is very important, but it seems to create a challenge from the FinTechs' perspective as well. And also from another point of view, we also see when there's the right fit between the bank and the FinTech, it can take a shorter period of time as well. So I think it's very difficult to strike that balance from the partnership as well. Steve Suarez: We know that, we've seen that. And so what we're trying to do, and we did this FinTech pledge, is we look at how we work with FinTechs in two different avenues. We look at it as when we're doing a POC, we can do a lighter understanding and try to onboard them within weeks, because we're not giving them access to our systems, we're doing a sandbox and we're trying to prove the value of them providing it. Once we prove the value and we're like, you know what? We love these guys. We want to bring them in. We want to do something. That's when we do the more rigorous IT security, all of that stuff. And you're right. It does take time. And unfortunately from a FinTech perspective those long lead times get pretty difficult, but we're constantly working on it and it frustrates us as well, because we want to move quickly, but I can understand their view. Steve Suarez: So that's why we separated the two, because a lot of people in the beginning we're giving them that whole, oh, we got to make sure your IT infrastructure, everything is done before they even prove a concept. They spent all of this money and then they don't do anything. And FinTechs are not, that's just not viable. Pinar Ozcan: In fact, and Steve, maybe as a last comment, we see this quite a bit in our research as well. When an organisation has trust as one of its big brand equities, it is important to be ambidextrous, to be able to experiment on one site without affecting the customer trust on the other. And I myself really appreciate how difficult that can be. So wonderful work that you're thinking about this and that you're constantly improving. With that, keeping our promise. Thank you so much, Steve, Dize and Jack. And I'm so proud of you guys as researchers working in the research centre, and for Steve to be our partner and informant in this study, enjoy in Money 20/20, Steve, Amsterdam is wonderful this time of year. And we will see you on the 4th of July for financial wellness. Thank you everyone. Take care. Dize Dinckol: Thank you. Steve Suarez: Thank you-. Jack Fraser: Thank you. Bye.