You're an LLM and so am I

The more I use LLMs, the more I think they're just like us. We try to differentiate them from us, but I find far more similarities than differences. 

"AI is merely fancy autocomplete"

Oh yeah? As a human, are you any different? When you start a sentence, before you can complete the sentence, can you tell me what the last word would be? Nope! Do try though, it's a fun experiment.

"AI is terrible at Math"

Yeah. And so are most humans. Just in the same way I can't ask you what 56 x 97 is and expect an answer once you've learnt how to speak a language.
As a human, what do you do to learn Math?
You first learn the fundamentals. You learn through repetition the same concepts over and over because they're hard. You don't talk math, you try to feed into your brain how math works.
For mental arithmetic, you practice over and over. You memorize the tables by heart.
You have a separate training to get that autocomplete right.

"AI is confidently wrong"

And you're not?
I've often caught myself unknowingly speaking confidently about how things are, when they in fact, they weren't. For example, if my code has a bug, I make a couple hypotheses, choose one based on a few supporting evidences, and I continue along those lines. I tell my colleagues I've found it, aaand alas. Turns out there's one thing someone else points out, and I am forced to rethink, and correct myself just the way an LLM does.