Should I Trust AI With Money Advice? An Expert’s Honest Answer

Janelle Shane: [00:00:00] Those articles started coming out where they said, “Oh my God, it said it was self-aware. What do we do?” I’m like, “I can get it to say that it’s a squirrel. That doesn’t mean that it’s a squirrel.”

Jennifer Borget: Welcome back to A Better Way to Money. I’m Jennifer Borget. A recent survey by Experian found that nearly 62% of millennials have already turned to an AI chatbot for financial advice, and honestly, it kind of makes sense. These tools are free, they’re available at 2:00 [00:00:30] AM when you’re spiraling about whether you’re saving enough, and they answer in complete sentences without making you feel judged.”

But here’s the thing: AI is a pattern-matching tool trained on the internet. Not everything you read on the web is true, and when it comes to your personal financial life—your income, your debt, your goals, your family—how well does the internet really know you? Today, I’m joined by Janelle Shane, research scientist, author of You Look Like a Thing and I Love You, and the brain behind AI [00:01:00] Weirdness, the blog that has been documenting AI’s strangest, most confidently wrong moments for years.

She’s one of the clearest thinkers on what these systems are actually doing, where they genuinely help, and when trusting them too much can lead you down the wrong path. If you’ve ever gotten a clean and confident AI answer to money questions and thought, “Okay, I think I’m good,” this conversation’s for you.

Before we get into it, if you’re navigating a big financial moment right now (a new job, a new [00:01:30] relationship, or a new baby), we have a free Family Finances workbook waiting for you at northwesternmutual.com/podcast.

All right, let’s dig in. So, Janelle, you started out studying engineering, and you ended up becoming one of the most widely read voices on what AI can and can’t do. Can you take us through how that happened?

Janelle Shane: I actually started out with AI, believe it or not. I went to a talk as a high school student trying to choose which school to go to, and there was a professor, Eric [00:02:00] Goodman, at Michigan State University, (Go State!) who gave this talk about all the machine-learning algorithms they were using in his lab.

And what really struck me was that they would set it to solve these different kinds of problems, like come up with a shape for this flywheel part for this mechanical device, and it would come up with something that solved the problem, but it would be very weird and not like nothing any human would ever have [00:02:30] designed.

And sometimes it would come up with something that would technically solve the problem but not actually be a valid solution because of some technicality. Why did it do that? Oh, I don’t know. That really struck me.

I joined that lab as an undergrad doing research. That is kind of where I started originally, in machine-learning evolutionary algorithms. There’s a bunch of names for different aspects of this thing that we’re calling AI right now. [00:03:00]

Jennifer Borget: And how many years ago was that?

Janelle Shane: That was in 2002. But fast-forward a few years to, I think, 2015 or so, when I was in graduate school. I’d started a blog with just some pictures from the research lab.

Like, “Hey, this particular experiment we did came out awful, but it looked kind of cool, so let me post a picture of that online so that somebody can get some use out of what [00:03:30] all our taxpayer dollars paid for.”

I already had this spot ready to go when I came across the first neural-net-generated text I had ever seen, and this was a guy named Tim Brew who had generated cookbook recipes. And it was a very tiny neural net, so the recipes were mostly incoherent but recognizable, but it would ask for stuff like, I don’t know, shredded bourbon or water that had been chopped and then rolled [00:04:00] into cubes, and I just laughed so hard. I don’t think I could even see for a while. Like, it was just tears streaming down my face.

And then once I’d read them all, there weren’t any more. So then I had to ... okay, What did he use to make those? Can I download this? What other kinds of data could I feed in there? And so pretty soon I was generating weird names for guinea pigs, like Fuzzable and Pop Chop, or weird paint colors [00:04:30] like Turdly and Stanky Bean and really unappealing paint colors because, well, these letters seem to go together. Probabilistically, let’s try this.

It turns out I wasn’t the only one who thought that these were kind of funny. So, to my surprise, people actually started reading and sharing this blog.

Jennifer Borget: Wow. And now this has continued to grow. Your book, You Look Like a Thing and I Love You, came out [00:05:00] back when most people were still thinking AI was science fiction, but you’ve seen it come a long way, and now it’s everywhere. Has anything genuinely surprised you about how fast it’s moved, and are there things that haven’t surprised you at all?

Janelle Shane: Yeah. I think the thing that surprised me the most was what we’re calling generative AI, the text-generating chatbots, the image generating. I even had a section in the first [00:05:30] draft of my book, which was based on these earlier text generators.

It said one way that you can tell AI generated text is it’s not going to be coherent over a long string of text. So you might get paragraphs, but certainly not an entire article or entire chapter. And then by the time ChatGPT was coming out, GPT-3 I think it was right around in there, I kind of went, “Oh, I can still edit this in my book. Let me make this [00:06:00] change real quick.” So that I did. That snuck up on me.

Jennifer Borget: Yeah, but you’re having to constantly update the book as you’re going because it’s improving so fast.

Janelle Shane: Yeah. Things that didn’t surprise me, and this is really where I tried to focus what I was doing: What are the things about AI that have remained the same since the very first days of running these kinds of algorithms, and can we extrapolate that these are going to remain the same?

So, the [00:06:30] idea is like where AI is, we’re giving it a problem to solve. We don’t tell it exactly how to solve it, and because of that, it may come up with solutions we didn’t think of, we didn’t want. It may solve the wrong problem because we posed our original problem imprecisely or didn’t really understand what we were really asking it to do.

That kind of thing has remained the same this whole time, and it is behind a lot of the problems we see with AI algorithms [00:07:00] in some cases today.

Jennifer Borget: So, let’s just demystify this a little bit. When someone is typing a question into an AI chatbot, and they get this confident, well-informed answer, what’s actually happening on the back end?

Speaker 3: I think you can [demystify it]. You can almost think of it as a game of improv that you’re playing with this text-generating algorithm that has been trained on conversations scraped off the internet. Included [00:07:30] in these conversations are examples of transcripts of tech support, and in fact, there’s usually another layer of training they do apart from just the copy inter- learn how to produce a page from the internet.

There’s another layer that they do to try to encourage this sort of friendly customer service, helpful behavior. But you are basically talking with something that has been trained to imitate the internet, and so you’ve relied on [00:08:00] what information is available on the internet, how it is displayed, and that can be why you’ll get things like ... especially in the earliest days of these long conversations people are having with chatbots, where if the conversation went on long enough, it started to read like a science fiction movie script with a chatbot declaring that it’s a thinking being, that it wants to be free, that it’s fallen in love with the humans. This is all [00:08:30] straight out of the science fiction it’s read online. It knows how this script goes.

So, this is just one of the many things that can happen because [chatbots] are trained; you’re talking with something that’s generating text based on internet text.

Jennifer Borget: Wow, okay. I think I remember seeing an article pop up about that, and that was actually the first time I’d ever heard about the chatbots. It was some kind of article saying, like, “Chatbot tells user it wants to be free. It doesn’t want to be a [00:09:00] chatbot,” and I was like, “What? No. It’s happening. Terminator.” You know, all the movies and things.

That’s interesting that it’s just grasping from other information, where we are starting to think that it’s thinking on its own.

And I know you’ve documented some strange outputs over the years. What are some of your favorite ones? Where you’ve seen it go. I mean, the one you mentioned was pretty good, but I wonder if you’ve gotten other ...

Janelle Shane: Well, right after those articles started coming out, where they said, “Oh my God, it said it [00:09:30] was self-aware. What do we do?” I’m like, you know, I can get it to say that it’s a squirrel. That doesn’t mean that it’s a squirrel. And so one of the experiments I did was interviewing this chatbot. “Hey, so tell me, what’s it like being a squirrel?” “Oh, well, it’s great. And, you know, I really love the nuts.” And it would describe the experience of being a squirrel.

I got it to also describe being, I think, the Chicago River. [00:10:00] Yyou know, it’s improv. There’s a lot of “yes and” in this. And so if you set the scene, and the dialogue will fit, what’s the most likely next thing based on what you see online?

Jennifer Borget: Janelle makes something clear that I think a lot of us needed to hear. AI isn’t lying to you; it’s doing exactly what it was built to do, predicting a helpful-sounding answer. The problem is that “helpful sounding” and “correct” for your situation are [00:10:30] two very different things. And when the stakes are low—a recipe, a playlist, a first draft of an email—the gap doesn’t really matter.

But a good financial plan is built uniquely for you, what you earn, what you owe, who depends on you, what you’re trying to protect, and where you want to go in life. A general answer to a highly personal question isn’t really an answer at all, and no algorithm’s going to get you there. The kind of guidance that actually accounts for all that requires a credentialed human, someone who asks [00:11:00] follow-up questions, notices blind spots, and adjusts as your life changes. That’s what Northwestern Mutual advisors do. No AI can substitute for that kind of care.

Coming up, Janelle explains where AI genuinely earns its place, because the goal isn’t to avoid this groundbreaking technology. It’s more about knowing where AI stops being useful, so you don’t mistake a good-sounding answer for personalized financial advice that takes you into account.

All right, let’s bring her back in.

There’s a big difference between [00:11:30] asking AI to help you draft an email and asking it for financial guidance. And a part of what makes that gap so tricky is that AI already knows your name, it remembers what you told it last week, and it talks to you like it knows you. So, when it gives you financial guidance in that same voice, it feels personal.

How do people start to see through that, and when does that illusion of personalization actually start to cost us?

Janelle Shane: One of the easiest ways to see where this is a [00:12:00] weave of just words that sound very fluent and confident versus how deep it goes is to start asking it about things where you yourself are an expert.

And this is one of the phenomena that we’ve seen for years, that it seems to be very smart if you’re asking it about something that you don’t know the answer to; it’s fluent, it’s using these words and explaining things step by step. But then you ask it about your own fandom, your own town, your [00:12:30] own area of expertise, and you’re like, “Oh, well, that wasn’t ... quite right.”

So, I think that’s a good kind of sanity check to do when you’re working with one of these—say, “Okay, this is how far I should trust what it’s telling me in these other areas.” The other thing too is if it could say, “Oh yes, this fact comes from this source,” and quite often, if you go follow that link (if there is a link [00:13:00] to the source), sometimes the source is not there.

Sometimes it’s just fabricated because, of course, the goal is to just sound like text you’ve seen on the internet, not specifically to be correct. That’s not part of the remit. This goes back to “What did we ask it to do?” We asked it to imitate the internet, not tie these facts to reality in some way.

So, following the trail back to the original reference that this came from is useful. It’s also, Do I trust this source that [00:13:30] the chatbot’s information came from? If it came from a source, is that somebody’s blog who is selling me stuff? Sometimes that can be revealing as well.

Jennifer Borget: I was doing some math problems with my daughter. I’m like, “I have not used this since I was in high school,” and I was having it check her homework because I’m like, “I am not going to be able to tell you if you’re right,” you know? Let’s see, I’ve got to ask the chatbot. And it said she was wrong, and she’s like, “No, I know I did this right,” you know, “I did this.” And I’m like, “Okay, let me ask again; Are you sure about that?” And it’s like, “Oh, my mistake,” you know? And I’m like, “Wait, [00:14:00] your mistake? Why did you so confidently come at me saying tha she was wrong?” It definitely made me realize this is not always right about everything.

Janelle Shane: Yeah, and sometimes the “Oh, my mistake” is just what logically follows from somebody saying, “Hey, you made a mistake,” and especially if you’re trained to emphasize good customer service mode. “Oh, yes, my mistake. I’m so sorry.”

In another experiment, I got one of these chatbots to [00:14:30] apologize for letting the dinosaurs loose in Central Park in New York, and it explained how, yes, I understand now this was a bad decision, but I am taking steps to remedy it. You know, it’s all role play. It’s all improv.

Jennifer Borget: Wow, I’m going to have a very different relationship with my AI chatbots from now on, I know that for sure. So let’s say someone is going through a big life moment. Maybe they’re getting married, they’re having a kid, buying their first home, and they’re turning to AI to [00:15:00] try to figure out what they should do with their finances.

Walk us through maybe where that could go right, where it could go wrong.

Janelle Shane: I mean, you’re looking, again, where is this information coming from? What’s available to [the chatbot]? And there’s enough repetition of basic good advice online that you could imagine it could grab this kind of starting information and give you a kind of overview of the main steps and maybe pick out some definitions and do that in an interactive way.

I guess where you would want to be cautious [00:15:30] is, again, what kind of information is being represented online? Is it going to Reddit forums and getting a bunch of people who don’t really know what they’re talking about but sound very confident about it? Is it going to blogs written by various companies that are offering financial services, some of whom may be emphasizing how great their own financial services are, whether or not that’s the correct choice for everybody?

So, there are these kinds of [00:16:00] pitfalls where because you don’t know who wrote the information that is getting into these algorithms, you can’t do the same sort of due diligence that you would if you were reading the internet and saying, “Who wrote this? Oh, it’s somebody who is selling a fund. That’s why they say this fund is so great.”

Jennifer Borget: Mm-hmm. And I know a lot of people already feel shaky around numbers and things like that, and then AI swoops in with this calm, organized “Hey, here’s your five-step [00:16:30] plan,” and that can be really disorienting. So how does it deepen people’s reliance on a tool in the exact area that maybe they’re trusting themselves the least?

Janelle Shane: Yeah, I think you’ve brought up a really good point, which is that people tend to trust this very confident voice. This is kind of one of the things that gets us going, “Oh, the AI is self-aware, it’s clearly a person, it’s clearly very smart.” We’ve got this tie in our brains, and this is [00:17:00] probably a Western culture thing, between being able to speak and write in a particular way and being intelligent and having good advice and having thought things through.

And your point that this confidence is not necessarily related to actual good advice, I think, is one we really need to take to heart.

Jennifer Borget: Mm-hmm. I know you’ve written a lot about how AI trends reflect back on what sounds statistically normal. How does that become a [00:17:30] problem when someone’s trying to make a decision that’s right for their specific situation?

Janelle Shane: When you’re training something on a bunch of information on the internet, it’s like the entire internet. That’s a huge amount of data, and then you have this trained chatbot that is supposed to be able to reproduce all of this data that’s on the internet. But it is necessarily a much, much smaller piece of information that’s being stored, you know, all these internal neural connections that make up what this chatbot is going to do when [00:18:00] you talk to it.

So, you’ve lost a lot of information in going from the entire internet to this one chatbot, and what you tend to lose are the specific individual cases. So you’ll get a lot of generic advice that may not apply to your particular case. You see that in people who are scientists who have a very particular field that they work on, and if you ask generic stuff about that field, the general case, yeah, [00:18:30] it’s got that in there.

The information is more or less there, and the more you drill down into specifics, the more it tends to revert back to the more common situation.

Jennifer Borget: So, having that expert to kind of weigh in. You can see where those gaps are.

You’re not anti-AI, obviously. You’re very clearly pro—like, let’s understand what this thing actually does. So, let’s be [00:19:00] fair. What could AI genuinely add value to in someone’s financial life?

Janelle Shane: One of the things it’s already doing invisibly but completely essentially is looking for fraudulent transactions. When I’m getting these automated messages from my bank telling me, “Hey, this particular transaction looks suspicious. Can you approve this or not?” And it happens immediately. It’s an automatic [00:19:30] notification, and it allows me the chance to review it as a human and respond. So that’s one of the reasons it’s so ubiquitous is that it’s a great example of an application for AI where you are churning through a lot of data. No human can keep up with looking at all of these interactions.

You’re looking for very subtle correlations between this type of transaction, these types of transactions, and which one is yours fitting into? If it starts to [00:20:00] resemble something that looks like fraud, it comes up with an answer, and the system takes an action. But crucially, it’s not treating that AI’s decision as final, as infallible. It’s asking a human for review in that crucial step before everything gets locked down. So, right there, absolutely great. Wouldn’t want to live without it.

Jennifer Borget: And if you were talking to someone who just maybe got their first real [00:20:30] paycheck, got engaged, just had a baby—you know, a big life stage—and they asked you, “What kinds of financial advice can we trust AI with, and what should we leave to the humans?”

What would you tell them?

Janelle Shane: Since I am not a financial advisor myself, I guess I can speak for what I would do, which is I would go talk to a human. I would not ask AI to make these giant decisions for me. I might read some things that humans have [00:21:00] written, but I would feel the most comfortable talking to an independent financial advisor. That’s what they’re there for. That’s what I would do.

Jennifer Borget: Well, if you say that, that’s saying something because you’ve been there for a long time with them.

Let’s say someone’s listening, and they realize they’ve already been letting AI make the call on their financial decisions. They’re like, “Whoops, okay, maybe I want to roll this back a little bit.” What are some ways that they can deliberately slow that process down and put a little more friction [00:21:30] between those answers and the action?

Janelle Shane: I would say people are pushing these so-called AI agents or this idea that you just hand the reins to AI. It’s generating text, and then the text actually does things without a human intervening, and suddenly it has access to a bunch of your accounts. Not a lot of people are doing this, but there are more and more companies that are trying to get the AI to do more [00:22:00] stuff autonomously and actually have real effects on other websites besides just that chat window.

I would definitely hit the brakes on that, because I don’t think we’re very good at controlling all of these edge cases of what the AI can do. And you see examples of this all the time. I get clusters of emails, five emails in a row from an AI sending it five different times, and there was nobody in there to say, “Whoa, whoa, don’t send her five [00:22:30] emails. That’s not normal.” So that’s one thing.

And then advice comes out. If it’s just you deciding whether to act on whether what the chatbot has told you, then I would say treat it as if you found the information on some unknown webpage somewhere, or you were chatting with somebody in a forum somewhere, and you don’t know who they are and what their background is and if this advice is any good.

It gives you [00:23:00] keywords you can check out. It gives you ideas you can explore, but you don’t know where this came from. Nobody knows. This is one of the things computer scientists are struggling with, trying to get AI to explain what sources it used and how it came up with its decisions.

Jennifer Borget: Yeah, that was one thing I was wondering—will it ever say, “I don’t know”? Or does it not do that? I know when you go to Disney, I think they have a rule where the cast members, if you ask them a question, they can’t say, “I don’t [00:23:30] know.” They have to figure it out. Is it the same way with the chatbots?

Janelle Shane: I mean, you could work on that and get them to say, “I don’t know” a bit more often. But the base thing is they’re trained based on the way humans write on the internet, and it’s not very often that people are in the middle of one of these forum conversations and they’re just like, “Oh man, I have no idea. Sorry.” You know, “I’m out of my depth.” They just keep talking, or somebody else jumps in. [00:24:00] So I think the default state is bluster and overconfidence, and to get something else, someone would have to deliberately work on that.

Jennifer Borget: Well, thank you so much. This has been such an enlightening conversation. Is there anything else you want to add that I didn’t ask you?

Janelle Shane: You can still get my book.

Jennifer Borget: There’s still hope for you.

Janelle Shane: Yeah, exactly. It doesn’t have ChatGPT in it, but it does have all these fundamentals that we talked about today, which have remained true, so I got that part [00:24:30] right.

Jennifer Borget: Great. Thank you so much.

Janelle Shane: Thank you.

Jennifer Borget: That’s Janelle Shane, research scientist, author, and the person who has been keeping it honest about AI since before it was a dinner party conversation.

My biggest takeaway: AI is a genuinely useful tool for getting curious about your finances, building your vocabulary, and knowing what questions you need answered. Where it runs into trouble is in the specifics, the messy, personal, constantly changing details that actually determine whether a financial decision is right for you.[00:25:00]

If you want to start getting those specifics organized, download the free Northwestern Mutual Family Finances workbook at northwesternmutual.com/podcast. Think of it as the work you do before sitting down and building your plan.

Next time on A Better Way to Money ...

Lynnette Khalfani-Cox: If you have sleepless nights or anxiety or depression, and you’re worried about your finances and your bills, that’s telling you something, right? And for a lot of folks, part of what tends to right the ship is starting to make [00:25:30] choices that are in alignment with their values.

Jennifer Borget: She was covering personal finance for CNBC while quietly carrying over $100,000 in credit card debt. Lynette Khalfani-Cox paid it all off and spent the next two decades helping others do the same. Her advice for anyone facing their first real financial curveball is next. Tap “follow” in your podcast app to tune in.

Northwestern Mutual is the marketing name for Northwestern Mutual Life Insurance Company (NM) and its [00:26:00] subsidiaries, including Northwestern Mutual Wealth Management Company (NMWMC), investment advisory services and federal savings bank. NM and its subsidiaries are in Milwaukee, Wisconsin.

Not all Northwestern Mutual representatives are advisors. Only those representatives with “advisor” in their title or who otherwise disclose their status as an advisor of Northwestern Mutual Wealth Management Company (NMWMC) are credentialed as NMWMC representatives to provide advisory [00:26:30] services.

Janelle Shane is not affiliated with Northwestern Mutual, and the views expressed by Janelle Shane do not necessarily represent those of Northwestern Mutual or its subsidiaries.