If you’re more of a reader, below you’ll find the transcript of our conversation, edited for clarity. You can find our other Workday Podcast segments here.
Josh Krist: I've always wanted to write a news article about how many algorithms impact my day-to-day life: 10? 100? 1,000? My fascination with how technology changes my day and the world of work is why I'm very happy to have the following two guests on the Workday Podcast today.
Ajay Agrawal, professor of innovation and strategic management at the University of Toronto, is co-author of the book “Prediction Machines: The Simple Economics of Artificial Intelligence.” Sayan Chakraborty is the executive vice president of Technology at Workday, where he and his team have played a key part in weaving machine learning into the very fabric of our underlying platform.
I'm Josh Krist. I've had separate, extremely informative conversations with both of these guests so I am very excited that they had the time to come on the show together. Thank you, gentlemen.
Ajay Agrawal: Thanks, Josh.
Sayan Chakraborty: I have a question to lead us off. It’s been 18 months since the publication of “Prediction Machines.” I bought it the day it came out—which is a really short time in some industries but a really long time in the area of artificial intelligence (AI) and machine learning (ML). Looking back on the last 18 months, what do you wish you would have elaborated on more, or had delved into more deeply, in the book?
Agrawal: We’ve since learned a number of things. Even though 18 months is a short time to some, it feels like a very long time in this field.
If I was to pick one, the most salient is that we underestimated the impact on power. “Prediction Machines” is largely about what happens as the cost of prediction falls. The part we underestimated was when the cost of prediction falls, it can affect not just the way we do particular tasks, but it can affect the distribution of power. We are starting to see some organizations and countries beginning to get ahead of others in a way that may be hard for others to subsequently catch up. Because prediction, in some cases, confers power. And because of the way AI works, they learn. An organization’s ability to deliver better predictions will attract more users, more users generate more data, more data generates better predictions. Once that flywheel starts to turn—we had underestimated in “Prediction Machines” the impact that prediction has on power.
Chakraborty: So the rich get richer, and we end up in a situation that feels like “the haves and have-nots,” potentially.
Agrawal: It can go both ways. If we take an example like AI that runs search, we used to have power distributed evenly from coast to coast in terms of information curation. We called those centers of power “libraries.” When we wanted information, we would go to our library and there were people skilled in library science and there were librarians, and every town had a library—or several.
Now, everyone’s information is curated by one of two companies. The majority is one, in Mountain View, California. So on one hand, there's been an incredible concentration of power; all that power once distributed across all these libraries is now sitting in one organization. On the other hand, the access to information has been democratized in a way that is much more evenly distributed than before.
In lower-income countries, in lower-income neighborhoods that had less access to well-funded libraries, now that access is much more broadly distributed. The power is much more concentrated, in terms of the control, but access is very distributed.
Chakraborty: Right, it's almost two kinds of opposing forces. You've got this level where the general expertise and knowledge available to the average person is so much higher than in the past. Yet we, probably for the first time since the Library of Alexandria, have a sudden concentration of knowledge in a handful of locations.
Agrawal: The curation of knowledge. In other words, if you and I want to search “What are all the different ways people are using Workday?” we would most likely go to Google and search that, and most likely would get a bunch of results on how different people are using the product. They curate all that information.
That seemed very benign in the past. We are seeing now, as people take more notice of the power that comes with that kind of information, the U.S. Department of Justice and the European Commission for Antitrust are starting to worry about the power distributed across two countries, the U.S. and China, through the concentrated information curation of Baidu, Google, and Microsoft Bing.
Krist: And I know you said there were three things that you were wanting to update or wanting to speak to. What were those others?
Agrawal: Another one is the relationship between prediction and judgment. In the book, we spent a fair amount of time on the basic idea that prediction is what AI does, and judgment is what humans do.
As we’ve watched systems evolve and take judgment questions from readers, the question often comes up: How stable is human judgment? In other words, when AI gets enough examples of watching humans exercise their judgment, can AI just predict judgment? The answer, in many cases, is yes. In some sense, judgment is what humans use to inform decisions under conditions of sparse data.
Where we have learned more since we’ve written the book is how things evolve—something like driving—arguably there’s a ceiling to how far we can go in the sense of imagining a world that’s fully automated—we don’t need any human judgment. There’s many other things where the way we do things today is not the ceiling. We can do much better. If we think of healthcare 10 years from now, we’ll look back and say, “Wow, I can't believe we put up with such a primitive form of healthcare.”
And then the third thing since the book: We have started to recognize different categories of AI initiatives. What I call short-term, medium-term, and long-term. Short-term initiatives simply take prediction problems that we’re using predictive analytics for and we apply AI to get a performance lift. Things like at a bank, anti-money laundering sanction screening, fraud detection, and “know your customer.”
The second category are initiatives that we didn’t use predictive analytics for—we used humans. These are what we are converting into prediction problems—things like translation, driving, determining credit scores, replying to emails.
Then there’s a third category, which is rather than thinking about automating tasks, think about redesigning the whole way things are done, such as completely autonomous transportation systems.
The first case generally has a pretty short time to show some results and generate an ROI for the companies that deploy them—the ones that just replace predictive analytics. The second phase initiatives are longer-term. They take, in general, two to five years. The third type of initiatives are much longer-term.
Chakraborty: I think we've seen that an essential part of adoption is rethinking the user experience. In the enterprise user experience, some software is not well suited for the augmentation that comes from AI and ML.
Providing people with a better understanding, a higher-level understanding, before they dive into the detail has proven essential to make it usable to people so that they understand what they’re getting. I like to describe the experience to my team in this way: I want our users to be put into the right landscape rather than just led to the destination, so that the final steps can involve, to your earlier point, their judgment.
Agrawal: What do you view as low-hanging fruit, in terms of your area of application? What’s the most impactful use of machine intelligence?
Chakraborty: Where we are, with millions of users using Workday every day across thousands of customers, is bringing to bear, more uniformly, expertise across the enterprise. Many enterprises traditionally have relied on that expert sitting in that corner for judgment. And their judgment, through a combination of experience and skill, is dramatically better than everybody else.
And because that expertise is not well distributed, we get optimal decisions over here, and we get lots of sub-optimal decisions being made everywhere else because one person can only be in so many places. By bringing the ML to all the people in those decision-making locations, and uplifting their level of expertise—not all the way to the expert level, but still quite a bit higher—what we’re seeing and what we’re hoping to do is provide a lot more efficiency and a lot better, more optimal decision-making across the enterprise.
Agrawal: Can you give an example of one area of expertise or one decision that is being enhanced through this approach?
Chakraborty: We’re launching a feature called Journal Insights that is for our financial customers. It looks at all of your journal entries, all the transactions that make up your business. When you close your books, what you’re looking for is mistakes, for fraud, and that can take a substantial amount of time each quarter. People will sub-sample that data, they’ll look at subsets of the data so they get a statistically significant amount of data and then say, “Okay, we’re pretty good.”
Journal Insights allows two things. First, it is able to process data automatically as journals are created. You never get the bad data, if you will, into the system. The second, and probably the more intriguing one, it is able to learn the patterns of experts. A lot of things you or I might think are off, an expert may say, “Oh, no, that happens all the time. This time of year, that’s completely normal.” And they know this because they’ve been closing books for 20 years. But when that person retires, that is a huge loss to the enterprise in terms of someone just knowing off the top of their head that this set of things that looks like a potential problem really isn’t, and vice versa. Something that we might think is innocuous could be a real problem to go and dig into.
We are able to lift the signal out of the noise of these thousands and thousands—millions, really—of journal entries for our largest customers, and allow anybody who happens to be involved in closing to identify the salient points to dig into. Going back to the issue of the individual in this—what does AI and ML really mean?
Reading some of what you’ve done since “Predictions Machines” was published, I think we sit at a point where there is still a lot of hype, there is still a lot of concern, and it would be very interesting for me to hear you reflect on the concern side a bit more.
Agrawal: There are various areas of concern—things like privacy, bias, and what the impact will be on jobs. Those are the three big areas. With respect to privacy, it seems that on average, there is a trade. Obviously, AI is run on data, so at the individual level, people need to decide what they’re willing to give up from their data. The more they give up, the more work AI can do for them in terms of personalization.
For companies, they need to make decisions on where they draw the line, on the data they will use and the data they won't. That often gets confused with the data they will share with third parties. I think the former issue is the one that really defines the relationship between the customer and the company they’re exchanging data with.
The second issue—what that company will then share with third parties—should be, in my view, one that’s very explicit but is somewhat separate from the company’s mission—but should be fully transparent to the user. At national levels—for example, a region like Europe that has a very distinct privacy regime—this has benefited that region in some domains. People are more willing to share information because they know it’s going to be treated in a more careful way, but at the same time, it makes it much harder for their companies to compete. Privacy is a topic where once only legal privacy wonks thought it was interesting, but everyone else thought it was dull.
Chakraborty: Those “I Agree” and “Your Terms and Conditions” that we all agree to automatically and scroll through, right?
Agrawal: Right, whereas privacy now is moving to center stage as a strategy issue and an innovation policy issue at a national level.
Chakraborty: We’ve taken a very directed approach to that at Workday. We built our ML system honestly, as an opt-in/opt-out system, which is technically hard. So customers opt in, and it’s very explicit which benefits they can expect to get by opting in which data. So, there’s highly granular decision-making around what data to share. And then the option to opt out, and for us to expunge the data. And then of course, in our case, being very explicit that we don’t share the data with third parties. That has been central to who Workday is for some time, but it’s gratifying to see the future come to us as people now say, “This is no longer a rubber stamp. This is something that we are actively worried about.”
Agrawal: That approach you just described—I suspect that even three years ago most customers wouldn’t have really appreciated the difference between that approach and some of the approaches others were taking. I think that increasingly, that will become the dominant approach as people start to recognize how important that opting in/opting out is, and how critically important it is in regards to sharing with third parties.
With regards to bias, I think what we are recognizing is that AI systems can be designed to either amplify undesirable human bias or they can be designed in a way to significantly reduce human bias. For example, there’s a great study done by a professor at University of Chicago and a couple of colleagues where they trained an AI, then they built two AIs. The first one was trained regarding the bail decision—for judges deciding whether or not someone is granted bail. It happens about 10 million times a year in the U.S. They trained the AI, and the first AI they trained on judges’ past decisions. When they ran this AI on data it had never seen, it performed in a way that was indistinguishable from a human judge.
The second AI they trained—instead of training it on what the judges decided, they trained it on actual outcomes of whether people that were granted bail fled or showed up for their hearing. The decision of bail is a prediction problem, and all you’re predicting is whether or not a person is a flight risk, which is totally orthogonal to whether they are guilty.
What was interesting about the second AI that was trained on actual outcomes rather than on judges’ decisions, is that it became superhuman. It became much better than judges, and when they ran it on data it had never seen, it was effectively able to—holding the crime rate constant—reduce the number of people that were incarcerated by 40 percent. And furthermore, it disproportionately reduced that for some ethnic minorities. In other words, it was a useful exercise in demonstrating that depending on how the AI is trained, it can either adopt human bias or improve upon it. The meta-level point here is that there’s some important work being done toward demonstrating the ability of AI to improve upon an issue that right now people are very worried about.
The third issue, on jobs—this one is tricky. Everybody takes one of two extreme views: first, that AI is coming to take everyone’s jobs, so there is great fear; and second, that AI is going to enhance people and make them superhuman, and we will all be better off. I suspect that the answer will ultimately be both. It will be more of the first in the short-term because as AI moves into jobs, it will be very hard to retrain people, particularly after a certain age. So there will be some potential hardship. In economics, they use the euphemistic term dislocation. I think that can be quite a significant problem.
But, longer-term, there is this substantive upside benefit. The key is going to be distribution. How do we make sure that it’s good for everybody and not just for the beneficiaries of enhanced technology?
Krist: We have time for one or two more quick questions. Sayan, you said you bought “Prediction Machines” on the very first day. What was your big takeaway?
Chakraborty: At the time the book came out, there were either a lot of highly technical books for practitioners, and then a lot of almost breathless hype books. Either what I call “AI eschatology”—the destruction of the world by AI—or the future perfect world generated by AI. There wasn’t any information about the practical implications for individuals or for companies, and then this book landed.
It was a breath of fresh air because the book developed a framework for us to think, “Okay. What does this mean? As predictions get cheap, what happens next?” That allowed me to drive particular focus on what we were doing at Workday—rather than spreading ourselves everywhere—how we could provide benefit for our customers and for their employees.
Krist: Soaking up those pools of uncertainty, helping raise the level of decision-making, right?
Chakraborty: That’s right. In some sense, because of the way Workday is—we’re both a transaction engine with people doing things in Workday, and a data engine where people have lots of data in Workday. That cycle is very explicit. Like, what is the data that went into me hiring somebody as recorded in the Workday Recruiting engine?
To your point, a critical issue as we talk about job loss: How do I get hired? What skills do I have? That's a decision. And again, to your point, that decision can be biased. So, that’s front and center, not just in the area of all the data we can take advantage of, but the codified decisions as represented by these transactions. That has really focused our efforts to say, “Let's really hone what we're doing here.”
Krist: Ajay, can you give us a quick preview of what you're working on? What will we be talking about next time we meet?
Agrawal: Sure. First, let me thank Sayan for his comments about the book. It’s really gratifying when people who are building things—Sayan has a background as an entrepreneur, he’s a builder—find our work useful. That’s very gratifying to us, so thank you for that. And, let me make one other point here. One of the things that’s so gratifying to us about Workday’s interest in our work is that, while some companies find the book useful and figure out clever strategies to compete and make their products better, [Workday agrees with us on] something that we care a lot about: social welfare.
Social welfare has a very specific meaning in economics, and while on one hand, this technology is likely to bring a lot of benefits to humankind, it can also cause some pain along the way because of job dislocation. One of the things that is very compelling to us about Workday is that effectively we think about the labor market as a matching system—and what we see in Workday is an opportunity for a first-order increase in the efficiency of matches. Matching people with things that need to be done. I’m sure that’s not the language that’s used at Workday, but in economics, this is, at a society level as opposed to the company level, a very important function.
I’ve been talking about machine intelligence to various companies, and I’m surprised at the number of companies—when I mention Workday—where someone calls out from the audience, “Oh, you know, we use Workday too.”
It just makes me think okay, each company is using Workday to make itself more effective and efficient, but at a macro-level, it’s also enhancing matching. Which, from a social welfare point of view, is of great interest to us.
So, to quickly answer your question, the thing that I’m working on now is this issue of power. How change in prediction leads to shifts in power.
Krist: Looking forward to it. I want to thank both Ajay Agrawal and Sayan Chakraborty for joining us today.