05/27/2026
WARNING - Long Post (but worth it!)
This week I almost ignored a real refund because it looked exactly like a scam. I started getting emails and texts saying I had a credit with a medical supply vendor I no longer use. It said I had to respond by a certain date (just 3 more days to go!). There was a link, and it was not even to the vendor. It went to a payment processor. Every alarm bell went off, so I did not click and picked up the phone instead.
That is where it got interesting. The system picked up, asked for some information, and then “someone” came on the line. Calm, polite, but too structured. The rhythm was off. Then it mispronounced my name. Not once, every time. That was the moment I realized I was not talking to a person. I was talking to AI. I kept going anyway. I explained the situation. It repeated things back in a way that felt close but not quite right, then told me I had a zero balance. I pushed back and asked why I was getting messages. It said it needed to check and put me on hold. Yes, the AI put me on hold. It asked permission to put me on hold while it checked on a few things and talked to a Supervisior. When it came back, it confirmed I did have a refund, walked me through the process, and about ten minutes later the money hit my account.
Everything worked, but something about it did not sit right. At no point did they tell me I was speaking to an AI system, and that matters. Not because the AI failed, it did the job. It solved the problem. It even followed the playbook like a trained agent. It matters because of trust. When you think you are talking to a human, you communicate differently. You explain differently. You assume a level of understanding, empathy, and accountability. When it turns out you are talking to a machine, that changes the experience.
AI is not the issue here. The lack of transparency is. If I know I am talking to an AI system, I adjust. I get more direct. I simplify. I do not expect nuance. If I think I am talking to a human and it is not, now I am second-guessing everything, not just the conversation but the company behind it. That is where the line starts to blur.
So here is the bigger question. Do companies have an obligation to tell you upfront that you are about to interact with an AI assistant? Simple, clear, honest. Or is it acceptable to design these systems to pass as human as long as they get the job done? Those are two very different approaches to trust. One builds long-term confidence. The other works until it does not, and we are heading into a world where this is going to happen more and more across customer service, medical, and financial interactions.
I am curious where you land on this. If the AI solves your problem, do you care if you were not told, or does that cross a line for you?
