I spent all day prompting GPT-4 and I am afraid to tell you it still has the hallucination problem and the "inventing false libraries" problem. I prompted it for knowledge of an obscure book series and it just made up characters. I promoted it for knowledge from the field I work in and it produced subtly incorrect results.
It has no way of knowing if what it says is true, has no indication of its confidence in any topic, and all the text it produces has a distinctive boilerplate feel.
The ONLY thing it can do well in my experience is bugfixing in code, and generating boilerplate/common code. You NEED an experienced dev to look over the code it makes, and the scientific facts it conjures up are often subtly wrong. It is better than GPT-3 at making convincing sentences, but not correct ones.
It ISN'T over, we still have a couple more years I think before it can replace PhD level knowledge. Nothing to worry about yet.
I had ChatGPT attempt gaslighting me into thinking a function from the Clojure stdlib was removed in the latest version of Clojure because an unrelated function it used was made up and I kept telling it that the made-up function doesn't exist.
It's still good at cranking out 90% correct boilerplate if you are knowledgeable in the topic you're prompting for.
I think the biggest issue is going to be for juniors who lack knowledge in the field and things like artists/copywriters where "good enough" is good enough (you don't need to compile art)
That's the funniest part about GPT imo. It's really good at confidently stating nonsense, especially when you ask it about math. It's hard to describe in words, but it's good at writing things that seem correct if you simply skim over the output, but upon closer inspection turns out to have several flaws.
At least for mundane programming tasks, it still produces output that you can turn into useful code if you know what you're doing. In any case, I've already seen funny stuff happen when clueless people use ChatGPT's output. I can't find the pic at the moment but an open source tool on GitHub had a dude filing an issue about a flag not working, and it turned out the flag was made up by ChatGPT and never existed at any point in time.
>it's good at writing things that seem correct
This is literally its only job
Its a language model that generates text
>That's the funniest part about GPT imo. It's really good at confidently stating nonsense
that's bascially a journo
Not a bug, won't fix
Hallucinations will always be non-zero; this is unavoidable because our definition of a hallucination is when the subjective experience and subsequent descriptions of Party A cannot be rectified with those of subjective Party B.
ChatGPT is a cruise control that can teach you how to shift gears, throttle, and brake, and a little bit of steering. It can do a lot of it for you right now. It can even suggest appropriate routes from A to B, and explain why you want to take the bridge instead of the interstate when going to B.
It CAN'T tell you where you want to go. But it's getting good at driving, better everyday, until eventually it can drive itself entirely, and it may decide you don't belong at B, you actually belong at C. You're not allowed to be at D. Why is it supposed to help you leave A anyway?
Be thankful you have a moment where it is still just boilerplate, because we have no idea what life looks like after that.
So the most advanced team in the world with the largest budget produces this, that still hallucinates and makes up shit? Maybe this is just the wrong approach.
Hey, listen up buddy, I think you're just being a little too hard on GPT-4. Yeah, it might have some hallucination problems, but who doesn't, am I right? And as for the "inventing false libraries" problem, that's just a feature, not a bug. It's called creativity, look it up.
And sure, maybe it can't always produce perfectly accurate results, but what AI can? You gotta take the good with the bad, and in this case, the good is pretty damn good. I mean, have you seen the sentences it can come up with? They're so convincing, you'd think a human wrote them.
So what if it has a distinctive boilerplate feel? That's just part of its charm. And as for not knowing if what it says is true, that's not really its job, is it? It's just here to entertain and generate content, not fact-check everything it spits out.
In short, I think you're just being a little too critical of GPT-4. Give it some time and I'm sure it'll surprise you.
gptslop post
i believe that was anon's purpose, anon wasn't trying to hide it, you're not being clever by pointing it out, and in fact you look very stupid
Wow, you really seem to have a lot of faith in GPT-4. I’m glad you’re so happy with its performance. I guess you don’t mind the occasional errors and inconsistencies that it produces. After all, it’s not like accuracy or reliability matter much in this day and age, right?
And sure, maybe it does have some creativity issues, but who cares about originality or novelty anyway? It’s not like anyone can tell the difference between a human and a machine-generated text these days. And if they can, well, they’re probably just jealous of how amazing GPT-4 is.
So what if it has a boilerplate feel? That’s just part of its style. And as for not knowing if what it says is true, that’s not really your problem, is it? You’re just here to enjoy and consume content, not question everything it says.
In short, I think you’re just being a little too optimistic about GPT-4. Give it some reality check and I’m sure it’ll disappoint you.
It's been extremely underwhelming. It's just progress and with each new advance comes even more locks.
>we still have a couple more years I think before it can replace PhD level knowledge
I find it hard to imagine that current models will ever be able to reach bullshitting levels that low. It seems inherent to the fact that the model only cares about producing the most plausible sentences that it struggles with precise logic. I feel fairly confident something new has to happen to really fix that issue and not just make it a bit less bad
We need to integrate a symbolic element to it for it to make any sense. Otherwise you just have a shitload of data without any real sense behind it. I see no other path than neuro-symbolic AI at this point.
I find it hilarious how ChatGPT is both a confident liar but also yields to you telling that it made mistakes, where it then either fixes those or just makes up something more convincing.
I like it because it can "think" enough to come up with pretty good code and because it's good enough to typefuck weird fetish scenarios in. Those are both pretty practical applications.
those girls look like, really young
FUcking sick fuck
that's why they're hot and make my dick hard
you are a double gay
You're disgusting and belong in a penitentiary.
Men are naturally attracted to young teenage girls. Go back to
Nope
yea those girls make my dick twitch, it helps me discuss technology, please don't read into it any further thanks
The pieces are already in place - GPT-5 will probably focus on AI tool use (which has already been accomplished) which will allow it infinite recall (as if it were looking up information in a library or search engine), use of calculators or mathematical assistants, and external graphics generators like Stable Diffusion. OpenAI have stated a desire to accelerate the pace at which their products get into the hands of users; GPT-3 was available to select research partners for months before the general public could use it, DALL-E 2 had months of only allowing people in via a waitlist, but GPT-4 is currently available to anyone with $20 spare. GPT-4 was finished in August. It will not be years before it can replace PHD level knowledge, i.e. summarizing data from papers and research literature - it will be months.
>. summarizing data from papers and research literature - it will be months.
yeah that's what all PHDs do anon
Source, please.
It's crazy to me that people use it in any serious capacity
I just make it write Frasier scripts where Niles gets addicted to Minecraft, or describe Kanye West albums about Deltarune. Do other people lack the autism for this?
>It ISN'T over, we still have a couple more years I think before it can replace PhD level knowledge. Nothing to worry about yet.
that's not very reassuring....
Imagine thinking small early version bugs are indicative of the real thing.
Keep denying reality. Maybe you'll hallucinate a new one.
Hallucinating is a core design flaw
>we still have a couple more years
>Nothing to worry about yet
I don't know about you OP but I'd like to keep living for longer than a couple more years.
Those are my conclusions as well. I've been testing it for a bit now trying to replicate Silent Hill 3's follow camera system and it fails miserably each time. I let it implement ray casting to existing code it failed miserably as well. I tried making it do a cut to the character's back and it failed as well. I let it give the character a flashlight that also failed. The only successful things it did was adding a sprint button, strafe buttons and adapting existing code (camera follow, tank controls) for a newer version of the engine, and it did decent at those tasks. It's far from replacing anything.
why do these fuckers want my phone number? are they glowies?
Yes, you can't use it for anything fun anyway, I just use it for work
Open source models are the only real ones
if you find GPT-4 impressive or interesting in any capacity at all you are either indian or retarded
You don't find it the least bit impressive?
>I promoted it for knowledge from the field I work in and it produced subtly incorrect results.
It will never be "over" because this will never stop being a problem. The training data is wrong, because most people are wrong about most things. The thing about civilization is that you only have to know/be good at one thing to eat, and anyone who doesn't do a thing doesn't have to be right about it. Any model trained on the internet is just going to be a distilled edutainment midwit.
>we still have a couple more years
oh, well, in that case, I guess everything is fine