AI and it's effect on your music, your job and the future

[jaxadam]

A good AI should have warned you about horse girls.

This was back when I was still playing snake on my Nokia.

 

[budda]

I work in distribution currently. AI isnt about to load trucks for me, but if it planned our shipping routes my job may be easier .

someone correct me if im wrong, but theres enough automation available that most people could have more free time and work less *if thats how the system had been designed*

 

[TedEH]

most people could have more free time and work less

But that's COMMUNISM. If you're not sweating and breaking your back for my benefit, you're a BAD PERSON.

 

[narad]

I don't think we're ever going to agree on this point. I liken this to the "you can't prove God doesn't exist" argument. The burden of proof is on the side making the claim. The claim is that AI is currently capable of "reasoning", and that is the part that needs to be proven. LLMs have the appearance of reasoning, because that's exactly it's purpose, that's what it's designed and trained to do, but so do a lot of things that can't actually reason. Saying that AI can "reason" is anthropomorphism.

The immediate counterpoint to "AI isn't currently capable of 'reasoning'" is to point at various reasoning benchmarks, where the bigger and more recent models perform better than older ones, and where current systems often outperform humans at reasoning tasks. Which gets to the point, either things are capable of reasoning to various degrees, or you are off in philosophy land where reasoning is or isn't there and you need to take it up with Searle.

As it pertains to this discussion, if we have a box that gives all the correct answers, it doesn't matter whether the box "knows" or "reasons" in some human-like way -- if you give it a set of reasoning based questions that it's never seen, and it answers them, hey, that's about the most relevant definition of reasoning you're going to get in terms of what jobs it's going to be capable of doing. If it's capable of doing that for domain X, and you work in domain X, then the fact that you might have some broader understanding of life outside of work that the machine does not, is not really relevant. i.e., you don't need strong AI to be superhuman or just make more sense in a company's bottom line calculations.

If you're not talking about reasoning in a practical and empirically verifiable sense, then you might as well just define reasoning as "things humans do". In that sense, I think yes, it is kind of like an ontological argument, but it's in fact the idea of 'we reason' or 'we know' that is comparable to 'god exists', in that you can't actually explain at all how a human brain knows or reasons about anything, but when you're given a machine that outperforms humans at tasks designed to assess knowledge or reasoning, that's immediately discarded as not being comparable to what we are doing because "statistics". Of course there are qualia that the machine does not have access to that are an important part of what we would consider "knowing" something, but the subjective experience of things is I think not something that's going to be relevant to having your job replaced by a machine that knows how to answer customer queries equally as well, and with infinite patience and availability.

And I think I mentioned this in the previous thread, but GPT-4 gets an 88% on the LSAT, which is full of reasoning questions. It's in the 90th percentile in most of these tests. It's not solving these by finding the answer online -- this is reasoning. And there are a number of academic datasets made for this explicit purpose, to assess systematically the extent to which such models can reason (and the many forms of reasoning). It should not be surprising that these papers do not begin, ~"As we all know, machines are incapable of reasoning. Still, we felt the urge to devise tests that we all know are inevitably in vain, because we have extra grant money".

If an AI sees an image of a pen, it has zero internal concept of what a pen is or what it could do. All it has is a bunch of trained weights that make it likely that a number of tokens in some order are a likely response to whatever the hell it might be looking at. Statistics dictate that if you've trained the weights to do so, you might get some text saying "this is a pen, you write with it", but you could just as easily end up with "this is a dog, which is a useful tool for calculating distance" if given the right data to work with - and the machine has no idea how ridiculous that is, because it has no idea what a pen, a dog, or "distance" mean. It's all equal and arbitrary and carries no meaning or consideration past the evaluation of tokens and some statistical model.

I don't really get this, but the LLM sees the world through text. If you were put a human in a world where there was an important relationship between dogs and distances, then humans too would learn these relationships. Humans are one part evolutionary bias, one part data-driven. But yes, if you were to in essence present a bonkers world to an LLM, it would learn a bonkers representation where concepts were not related in the way they are in our world.

I mean, there's 101 examples out there of AI or ChatGPT failing miserably at a lot of things, like math, or ASCII art, or understanding when a user is being sarcastic, or just taking everything at face value, giving non-answers, etc. The kind of stuff that would be unreasonable as an answer from an actually intelligent agent.

The best way to illustrate this is that a lot of the time, the answers you get from something like GPT have trouble inferring whether or not the answer being given matches the intent of the prompt. I asked about the pen above, and had to pry a few times to get details, asking how the pen got on the desk. Every answer I get is just a list of generic factors, much like the earlier example given, and insists that part of the way to the desk might be through things like magnetic attraction, or being blown there by the wind - which makes zero sense. That's not a reasonable answer. That's not an answer arrived at via reasoning. Because reason would dictate that I was looking for an answer like what I typed out above, or that you'd need further evidence to support weird environmental factors, but that's what it goes with because the tokens of [speculate] [pen] [placement] [lifecycle] are going to have associations with the generic non-answers of [direct placement] [environmental factors] [manufacturing] [transport].

View attachment 133368

But sure. "How is the pen likely to have ended up on the desk" is definitely magnets. That's reasonable.

Okay, so I'm giving you free reign to show me where the limits of ChatGPT's reasoning capabilities are. Failing miserably at math or ASCII art is not showing that it a model is incapable of reasoning, or the >85% of north american high schoolers who can't do math as well as ChatGPT are therefore similarly existing without the ability to reason. It has to be an example involving reason.

And so you choose this pen example and talk about how it's not reasonable to suggest that a pen could be blown by wind or by magnetic attraction to its location on the table. I agree these are not likely explanations. But I also noticed your screen grab started with #5, which, well, was kind of an indication of being cagey. Why not show the full list? So I asked ChatGPT myself and was not surprised that the explanations you cropped out were essentially the one you provided above -- that "someone left it behind" or "deliberate placement for future use" were the first two. But still, I asked for the model to assign some probabilities to each of these explanations, and I further added that this was an office setup and didn't have children or pets, since you as a human could likely infer this from seeing this hypothetical room. The following is what it produced -- 0% to wind, and no mention of magnetism.

The list looks pretty reasonable to me...

So where's the shortcoming here?

 

[Fenriswolf]

And that's a large part of why I don't think AI can do music. Music is so insanely subjective, and the criteria to evaluate what makes music, let alone "good" music, can barely be pinned down by people, so I can't imagine how you'd train a machine to recognize it.

I've just skimmed the thread, as far as Dr. Luke pop music, it will absolutely be writing it, it's already been shown that it's written to a formula.

As far as affecting my job, he might be a douche, but Justin Waller has a point.

I"m a couple months away from being a millwright, all AI is going to maybe do for me is make my job faster like lasers have.

 

[MaxOfMetal]

I work in distribution currently. AI isnt about to load trucks for me, but if it planned our shipping routes my job may be easier .

someone correct me if im wrong, but theres enough automation available that most people could have more free time and work less *if thats how the system had been designed*

Three of our plants have almost completely automated distribution centers. Robot forklifts, robot order pickers, robot lumpers, etc. and a computer system that assembles and maintains orders.

The only folks working are there to communicate to truckers and make sure stuff doesn't break.

This isn't new tech, they've been around for almost two decades now. I don't think it really counts as "AI" though.

 

[Fenriswolf]

Three of our plants have almost completely automated distribution centers. Robot forklifts, robot order pickers, robot lumpers, etc. and a computer system that assembles and maintains orders.

The only folks working are there to communicate to truckers and make sure stuff doesn't break.

This isn't new tech, they've been around for almost two decades now. I don't think it really counts as "AI" though.

I had a job loading trucks for a couple months. As someone who grew up watching Terminator and is very anti AI because Skynet, you want me to load a container like a pro solitaire player on a forklift that's so worn out it makes Mia Kalifa look like a virgin. You can either keep paying me like shit to load normal 18 wheeler trailers, or pay me like shit to do something that actually takes some skill and get what you pay for.

 

[narad]

The "Mia Kalifa look like a virgin" line is so worn out that it makes Mia Kalifa look like a virgin

 

[Fenriswolf]

The "Mia Kalifa look like a virgin" line is so worn out that it makes Mia Kalifa look like a virgin

I had to look up how to spell her name. I would insert joke here, but can't think of one subtle enough.

 

[budda]

Three of our plants have almost completely automated distribution centers. Robot forklifts, robot order pickers, robot lumpers, etc. and a computer system that assembles and maintains orders.

The only folks working are there to communicate to truckers and make sure stuff doesn't break.

This isn't new tech, they've been around for almost two decades now. I don't think it really counts as "AI" though.

Sounds like your company has money

 

[MaxOfMetal]

Sounds like your company has money

More than they like telling us.

Compared to people, they're cheap.

 

[Fenriswolf]

More than they like telling us.

Compared to people, they're cheap.

Just think of the pizza party you could get.

 

[thraxil]

I had to wait till the weekend to comment in here and I'm going to try real hard not to write an entire book because I do still have shit I should be doing.

Some notes on my background: first, my partner is a computer science professor at an art school and runs a program on creative computing and game design. She's been working with ML in that context for about 15 years. Originally a lot of stuff around training models to recognize gestures and use that to drive visualization, music production, etc. She created InteractML, a tool for adding ML functionality into the Unit game engine (that's a whole other topic though...) and in the last few years has been obsessed with Stable Diffusion, DALL-E, and LLMs and any other generative stuff she can get her hands on and has been thinking and writing a lot on how those technologies change the landscape for creative professionals.

Second, since March of this year, I've been principal engineer at an AI-focused startup. We're still in stealth mode so I can't be too specific about what we're doing, but we're definitely on the more "boring" side than what my partner does. We deal more with regulatory compliance, enterprise security, data privacy, and stuff that would put many of you to sleep, but got us some serious VC funding. Up to this year, I really wasn't an AI/ML person at all. I'd played around with markov chains back in high school in the 90's and some neural networks in college. At the time, both were fun to play with but not very useful for most applications. We just didn't have the computing power to do non-trivial things with them. I mention those because if you squint a bit, LLMs kind of look like they're just a mashup of markov chains and neural networks with a few clever tricks and then just taken to a massive scale.

So now a few points and observations:

* If you're a professional translator and your job is mostly just to take text written in one language and rewrite it in another language, you should be really concerned. Your job is already basically obsolete. ChatGPT does translation extremely well, even for very technical, jargon heavy, or niche topics. My partner's school has a bad habit of admitting a lot of foreign students that don't actually speak English very well who then struggle in their classes. They all just use ChatGPT now and it's been game changing. If you haven't seen this in action, try taking a paragraph or two of text, having ChatGPT translate it to Chinese or some other language you don't know, then take the output from that, start up a new session, and have it translate that back to English. If you've ever done that with Google Translate or similar in the past, the results were often hilariously bad. What you get out of ChatGPT won't be exactly one to one with the original but it will be reasonably close; more like someone just re-phrased the original a bit.

* LLMs are language models so they're especially good at language related tasks. My partner, as an American at a UK school has often gotten in trouble for being too blunt and direct. She now passes most of her emails through ChatGPT first with a prompt like "rewrite this in a British academic style" and it's made a huge difference. This sort of thing, cleaning up text, fixing grammar, summarizing text that you give it, etc, where the focus is on language itself, is right up its alley. If you're a copy editor or similar, you should probably also be worried.

* Prompting really is still kind of an art form and something you need to spend a lot of time with to get the best results. You have to understand the limitations and strengths of the LLM you're working with and when to use tricks like asking it to first explain a strategy that it would use for answering a question, then applying that strategy while reasoning "out loud". Then you might still follow up by asking it identify weaknesses in its own logic, etc. If you go into ChatGPT or DALL-E with some basic naive prompt it can be pretty easy to conclude that they don't work very well. Someone who knows what they're doing can probably get results that are night and day better.

* On the art side of things, this has been a major conclusion for my partner. If you're lazy or a bad artist and try to use DALL-E to generate stuff for you outright, you'll get boring, derivative work that won't impress anyone. The artists who have really developed a keen sense of aesthetic, critical theory, and learn how to use ML as a tool to supplement what they're doing are reaping huge rewards. Even if you are a talented painter working in concept art, you can't paint as fast as Stable Diffusion can generate images. The trick is to get good at using that to iterate through a lot of ideas quickly and use the eye and sensibilities that you have to sift out the good stuff and know what to prompt to push it in the direction you want. If you're creative and talented and have put in the work both to understand what makes art good and bad and to use ML, it's a powerful tool. From a commercial perspective, those are going to be the more valuable skills than, eg, being really good at drawing. The art world has already been through a bunch of revolutions like this eg, lithography, photography, photoshop, etc. It never ends art, but it changes things (especially if you're expecting to get paid to make art).

* It's still important to just treat the output like a suggestion that you get from some online forum. You can never really trust it and you need to verify things yourself. If you read something on the internet, it's obviously not necessarily true or accurate. ChatGPT is basically a condensed version of the internet and you should trust it just about as much. Similarly, we've kept the internet around because it's proven pretty useful and LLMs appear to be following the same path.

* I have over twenty years of programing experience and I now use both ChatGPT and Github Copilot daily. Right now, they each have sweet spots. For really simple, repetitive, boilerplate stuff, Copilot tends to be very useful. It feels more like just an extension of an IDE's autocomplete. Autocomplete tends to be more limited but (especially if you're working in a typed language), when it tells you there's a method or parameter available, it's generally going to be right. Copilot gets things wrong sometimes but also can be more useful in dynamic languages or picking up whole repetitve blocks. Meanwhile, it's impressive what ChatGPT can do, but it takes a bit more work to get good results out of it. I find it less useful on the main codebases I work on professionally because 1) I know the language, framework, etc. really well already and 2) the challenges tend to be more related to making things work in the context of a large codebase with a lot of implicit constraints, existing conventions, etc. that would take me a long time to explain to ChatGPT every time. Where it really shines is when I need to deal with something that I'm less familiar with and would otherwise be spending a lot of time reading documentation, searching online, asking in forums, etc. I still wouldn't *trust* it and it's on me to verify that the code it generates really works (eg, it regularly hallucinates entire APIs and makes very basic mistakes) and you still need to know programming in general and the domain you're working in very well to be effective. But it's a really fast way to make quite a bit of progress if I'm dealing with a new language or framework. At previous employers, I've had interns and junior developers and I've spent a lot of time carving out small tasks for them, writing up clear documentations and requirements, and then closely reviewing the code they produce. Often, I spent more time managing the intern/junior than it would've taken me to just do the damn thing myself. I'd say that ChatGPT is at least as useful as most of the interns and some of the junior developers and it's much faster.

* On the "AI's going to take your job" thing, in most domains, I think it's currently around that intern/junior level. You wouldn't trust them to do the whole thing themselves without any supervision because they're going to make some really dumb mistakes and they need a lot of guidance to get the most value out of them. But ChatGPT is like having an intern-level employee available for every area of knowledge work in the entire world all at once who also happens to be fluent in most languages, has no ego, writes clearly, is extremely fast, always available, and costs almost nothing. If you're experienced in your field, it's not going to immediately replace you, but you should be learning to use it well or risk being replaced or beaten by a competitor who does know how to utilize it. I'm more worried about long term effects. Entry level jobs are going to be the most affected at first (at a small startup, it was already hard for me to argue to our CEO that we should hire junior devs or interns and now that's even harder to justify). But if younger people aren't getting that experience, we're quickly going to see an aging workforce of experts who actually know the domain getting things down with LLMs while it gets harder and harder for anyone else to get their foot in the door and start developing that expertise.

 

[TedEH]

So where's the shortcoming here?

That you had to provide the reasoning in order for it to give a reasonable answer. it didn't arrive at "of course, there's no chance this would happen" on it's own, otherwise it wouldn't have suggested those things. YOU did that. Not the AI. You did the reasoning part. If there's a 0% chance that something would happen, then why would that be a reasonable answer to have provided to the question in the first place?

Which gets to the point, either things are capable of reasoning to various degrees, or you are off in philosophy land where reasoning is or isn't there and you need to take it up with Searle.

I do think you're operating off of a very loose definition of what reasoning is, given that your only criteria seems to be "can pass a reasoning test when compared to previous iterations of itself" but with no concern for the process, or what any of it actually means. Yes, the text is very convincing, because that's exactly what it's designed to do. But it's an illusion. In the same way that if you play a video game, you didn't actually bump into geometry in a map. The geometry isn't real. It's an illusion. It's a metaphor. It's a very clever trick.

That's why I tried to provide what I mean by it: A computer does not, and cannot, actually understand the concepts it's dealing with. Like imagine you open a bitmap in Paint. Paint doesn't "know" what a bitmap is. Paint is capable of handling a bitmap under the right circumstances- so you can say it's "knows" only metaphorically or symbolically. But it has no real understanding. You could feed it any data at all, and it will treat it the same way. If you've ever written code to open a file, a lot of times step one is to read the header of the file to see what it is, at which point YOU know (or rather, you're assuming) what you're dealing with, and you write the rest accordingly. The rest of the code is entirely working off the assumption that your first assertion based on the header is correct. The only reasoning that ever applied was the reasoning of the programmer who created the instructions in the first place. Software doesn't "know" things. Anything you can say a piece of software "knows" is an expression of the programmers knowledge at the time the software was written. Saying that "AI knows what a pen is" is like saying "a car knows what the road is".

So yes, I think our disagreement is a philosophical one.

 

[narad]

That you had to provide the reasoning in order for it to give a reasonable answer. it didn't arrive at "of course, there's no chance this would happen" on it's own, otherwise it wouldn't have suggested those things. YOU did that. Not the AI. You did the reasoning part. If there's a 0% chance that something would happen, then why would that be a reasonable answer to have provided to the question in the first place?

I did not provide any reasoning.

 

[TedEH]

I did not provide any reasoning.

Then what is this?

But still, I asked for the model to assign some probabilities to each of these explanations, and I further added that this was an office setup and didn't have children or pets, since you as a human could likely infer this from seeing this hypothetical room.

You had to provide so much context that you wouldn't need to give to a reasonable agent for it to not provide you with nonsense or vague non-answers.

Again - reason does not mean "answer", it's the process. Reason is coming to a conclusion by operating on the actual understanding of the information. AI does not do this because it doesn't understand the information. Acting on data is not proof of understanding. GPT is just, one word at a time, going "what token makes it through the big system of weights and math based on what I've done so far". At no point is the data interpreted outside of the likelihood for one token to appear after a given sequence of other tokens.

 

[narad]

Then what is this?

You had to provide so much context that you wouldn't need to give to a reasonable agent for it to not provide you with nonsense or vague non-answers.

"so much context" was literally: the room is an office with no children or pet factors. You're crazy if you think that's where the heavy lifting is being done in this exercise. I think it reduced the probability if those things from like <3% to 0%.

Again - reason does not mean "answer", it's the process. Reason is coming to a conclusion by operating on the actual understanding of the information. AI does not do this because it doesn't understand the information. Acting on data is not proof of understanding. GPT is just, one word at a time, going "what token makes it through the big system of weights and math based on what I've done so far". At no point is the data interpreted outside of the likelihood for one token to appear after a given sequence of other tokens.

Also, I think you have a big misunderstanding regarding emergent properties. Again, just because something's end goal is to predict the next word, says nothing about the mechanism about how it arrives there or what information is considered in the prediction process. Merely the fact that answers are structured and organized suggests that the answer exists in some sense internally prior to being generated word-by-word.

 

[TedEH]

Merely the fact that answers are structured and organized suggests that the answer exists in some sense internally prior to being generated word-by-word.

But the process of eking out that answer from the source data is not what I would call "reasoning". You're focused on the results, I'm focused on the process. You're focused on what emerges, I'm focused on why it emerged. What you're describing as reasoning, I think is just an illusion or appearance of reasoning. In that sense, I don't think what you've said is strictly true. Correct me if I'm wrong, but there's zero planning at each stage, at each word - meaning at no point does the machine hold some state that represents "the answer" it's trying to communicate. Again - where a reasoning agent holds a concept of the answer it wants to communicate, and then communicates it with you, an LLM simply calculates a bunch of likely words, and it just happens to be an emergent property of this system that it lands on reasonable-sounding answers a lot of the time. Up until the sentence has been fully generated, there was no answer.

Again, more analogies. If you have a math problem - say you're holding things in your left and right hands, and want to sum them - you can use a calculator, and the calculator will give you the right answer. You prompted it with something, it did *something*, and arrived at an answer. But there was no reason involved. At no point did the calculator understand what you were summing together, or why, or have any state that could represent an understanding of what numbers are in the first place. It's just a mechanical trick using gates. The intelligence was human, in the design of the calculator.

So is a calculator "good at math"? It depends on what you mean. If you mean that a calculator is a perfectly good tool for use in that field, then sure. If you mean that a calculator is capable of meaningfully solving for the underlying concepts, then no, it's not "good at math", because what it's doing isn't the larger concept of math, it's only calculating what you put into it - which is only a part of what constitutes math.

To me it feels like we're anthropomorphizing computers. Attributing a type of agency to them that they don't have, just because their behavior sort of mimics what a conscious agent might say.

Attributing that kind of reasoning to an AI, IMO, leads to putting much more trust than is warranted in the output. Like the lawyers who got fired for using ChatGPT instead of doing actual case research. Or the teachers etc. who were convinced that ChatGPT could identify if text was written by AI.

You can call what an AI does "reasoning" if you want to, but I can't, unless in a colloquial way.

 

[narad]

But the process of eking out that answer from the source data is not what I would call "reasoning". You're focused on the results, I'm focused on the process. You're focused on what emerges, I'm focused on why it emerged. What you're describing as reasoning, I think is just an illusion or appearance of reasoning. In that sense, I don't think what you've said is strictly true. Correct me if I'm wrong, but there's zero planning at each stage, at each word - meaning at no point does the machine hold some state that represents "the answer" it's trying to communicate. Again - where a reasoning agent holds a concept of the answer it wants to communicate, and then communicates it with you, an LLM simply calculates a bunch of likely words, and it just happens to be an emergent property of this system that it lands on reasonable-sounding answers a lot of the time. Up until the sentence has been fully generated, there was no answer.

Yea, that sounds incorrect. The whole point of the depth of the network is to make connections which are not immediately accessible in the surface form of the input, and to layout something like a plan or a compact representation of the input, from which the output is decoded. GPT-type architectures are decoder only, but that doesn't mean they don't have essentially an encoder built up inside them doing exactly that.

Maybe the clearest example would be in an image recognition model. It's trained simply to recognize what object is in the image, but in the forward pass filters of various sizes are identifying basic edges and textures, while others look at multiple of these and construct representations of particular objects. At one point some part of the network is looking at an internal representation of what is roughly an eye, and another eye, and a particular type of nose, and it starts to gain confidence that this is a cat face. But that recognition is not "flat" and purely statistical co-occurrence. It's hierarchical, and compositional with clear notions of important "concepts" within the task of image recognition, induced purely from the end task loss. Note that this is not all occurring on layer 13 or whatever -- various components are being recognized at various layers (which is part of why skip connections proved to be useful). It is similar in the text domain -- there are concepts which are not purely the text, and whose relationships with other concepts can sometimes be visualized to some extent, and where understanding of the relationships is evident in the predictions (unless you assume the presence of really specific training data). For instance, in your pen example, while it's not likely that a pen is on the table because of magnetic forces, the fact that pens are often made of metal, and metal is subject to magnetic forces, is a cute explanation that I would not have considered, and also an example of what I would call reasoning (literally a simple form of what's referred to as "multi-hop reasoning").

The arithmetic stuff is also interesting. The algorithm for basic arithmetic requires many steps, and each step must play out in a transformation in the depth of the network. It's clear I think that the model didn't memorize every addition/subtraction/multiplication you can do for some numbers greater than 4-5 digits, but you can still usually get the correct answer for these. As things keep going larger there starts to be more and more mistakes -- at some point the amount of steps in that implicit computation are simply greater than the number of "discrete" operations you can perform in a network of that depth, and things get weird. But some sort of basic algebraic computation engine was induced. That's also "reasoning". Of course we don't know which of the known algorithms (or something else) it is attempting to do.

But ya, it sounds like you think of the next word prediction as being based on some surface level co-occurrence of words in the input and the target words, but that's not how it works. Like right out you have an embedding space where concepts are oriented in the representational space of the model and the surface words no longer matter much (though the surface form is probably also represented in some dimensions to deal with things like rhyme). Then each further step allows for another round of transforming concepts in that space to something else, which is internally useful but we do not understand. But hypothetically if this is not reasoning, and it is just statistics, then why is depth in a network important?

And I guess it's also important to note that predecessor LLMs did explicitly hold a "state". Ultimately all the cramming stuff into a single result and then decoding was not as effective as letting the decoder do double-duty, and also dealt with problems like variable size inputs vs. constant state size.

To me it feels like we're anthropomorphizing computers. Attributing a type of agency to them that they don't have, just because their behavior sort of mimics what a conscious agent might say.

You can reason without anthropomorphizing. Prolog programs reason, they're just brittle.

Attributing that kind of reasoning to an AI, IMO, leads to putting much more trust than is warranted in the output. Like the lawyers who got fired for using ChatGPT instead of doing actual case research. Or the teachers etc. who were convinced that ChatGPT could identify if text was written by AI.

You can call what an AI does "reasoning" if you want to, but I can't, unless in a colloquial way.

I mean, you can say that, but you're kind of ignoring an entire field (and notably this book on my bookshelf that makes it pretty clear):

You can't blame idiots being idiots on people holding steadfast to the definition of reasoning as laid out decades ago.

 

[TedEH]

Admittedly a lot of that goes over my head. I'm obviously not an AI expert. Maybe I'm wrong. Maybe I don't have the words to articulate what I mean. I just have trouble wrapping my head around the idea that any machine can "know" something, past being a metaphor for some representation / interpretation of data that would otherwise be meaningless without the bespoke tool and code there to make something that looks like meaning from it.