Probably true, but they will only censor it after they kill the competition.. I would expect a fully open source solution too.. (I guess Yandex already opened their model)
>Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
a heavily controllable AI apparently, and they aren't investing nearly as much as microsoft did into chatgpt, a tiny fraction only.
>They already have the best AI people on their payroll.
Really? What makes you think that? The fact their gmail SPAM filter works like shit now? Or the fact they're buying some company no one has heard of so they can compete with another start up that's only been around for 7 years.
Google is just a censorship and advertisement company, that's all they do. Other than buy up other companies to try and be relevant before closing them down after losing in the market. Google has sucked for a long time and they're not getting any better.
Which, from the perspective of a Google exec, is NOTABUG WONTFIX. The point was, why would they feel the need to blow this money on some team no-one's ever heard about, which their existing AI teams mog to death?
Google invested in them because they noticed them buying so much Google Cloud Computing. Part of this deal means Anthropic (whose biggest investor before this was Alameda Research mind you) still has to keep buying Google Cloud Computing services. The CEO at least seems to be on the same page as Google, in that he does not want to give the public AI services they can use for themselves.
This is a weird weird weird weird one. How do they know Google isn't going to just rape and steal their AI?
>whose biggest investor before this was Alameda Research
Ah, I see how it is.
i don't understand why these people make inventions and then get scared about it. Just free it to the fucking public and stop being so egotistical about a machine doing things better than a human can. >hurr durr what if they take over the world oh no!!!
The world is already shit. Get over your gay little city lives and get real about your dumb movie fantasies.
It's 100% the same issue like with all big corps: inter company politics.
Any team that actually wanted to push commercialization has to fight against competing teams and stick their neck out in case something goes wrong.
Someone like anthropic, who isn't part of that mess, may very well be what's needed.
>They already have the best AI people on their payroll.
Really? What makes you think that? The fact their gmail SPAM filter works like shit now? Or the fact they're buying some company no one has heard of so they can compete with another start up that's only been around for 7 years.
Google is just a censorship and advertisement company, that's all they do. Other than buy up other companies to try and be relevant before closing them down after losing in the market. Google has sucked for a long time and they're not getting any better.
Just look at the papers released, Google is indisputably nr 1, the only reason everbody talks about OpenAI is because they're actually pushing forward with commercialization. They're not a leader purely research wise.
>just look at what paid PR posts are saying!
Get a brain, corporate dicksucker.
Google has never contributed even ONE thing to AI. Not one.
At least microsoft published several quality datasets that are still state of the art, greatly helping researchers. Google? Literally nothing.
Which, from the perspective of a Google exec, is NOTABUG WONTFIX. The point was, why would they feel the need to blow this money on some team no-one's ever heard about, which their existing AI teams mog to death?
[...] >whose biggest investor before this was Alameda Research
Ah, I see how it is.
There was a rumor that the Google leadership was absolutely shitting themselves when they saw how well chat GPT performed.
The main thing is that they realized it would be a one step drop in replacement for the average midwit to use instead of web search, which despite everything, is still 95% of their revenue.
a further is that it seems like google, what's believed to be a pioneer of AI behind closed software/doors, mighty be the first of the big tech corps to be truly fucked by it. we can only hope.
Google has never been able to do anything in the AI race. Deepmind was independent before google bought them (obviously) but they slowed down considerably as time passed past the acquisition (alphago was just around the same time as the acquisition and they didn't do anything after that beside reworking the same thing at every sauce, yet they had a few hot papers before that).
Google's main AI group was google brain, but they never achieved anything despite all the heavy hitters they hired.
welcome to modern alphabet who hasn't made a good decision in 8 years other than killing off shit products and only exists because they have an extremely profitable monopoly on ad serving
Google is a dead company. They haven't created anything new in the last 20 years. They are only staying alive thanks to what they already have, which they have a monopoly on. AI search will be the death of google search, since google never innovates their search. Even google reverse image search is complete garbage now because of google policies and if google ever adds AI search it will be garbage for the same reason.
Scroogle doesn't have anyone worth anything in AI outside deepmind, and they decided they wanted deepmind to be a product company focusing on the alpha line instead of an AI research company.
>Scroogle doesn't have anyone worth anything in AI outside deepmind
Their in house researchers literally developed the transformer architecture everyone is using these days, retard
Maybe I need corporate shilling here but genuinely what the fuck has alphabet been doing for the past like 10 years, everything they touch turns to hot garbage, with the exception of the pixel line and they still fuck that up.
Youtubers have been getting way too much money for way too long, and yeah there are a lot of based youtubers on there but realistically for a normie company that's actually a fucking huge problem esp since they barely get views , pewds, Jordan and gundam are so hated and are so spicy that it genuinely drives Normie's from the platform, killing off that type of content is normie company 101.
They fucked the search to the point that people started using bing of all things, chrome is slow as fuck so even normie's dont use the base version anymore , pixels break enough said, Chromebooks just died , their streaming service , dead, their social media attempts all dead, their ais all dead , and it seems to be a direct result of shitskinnifying the upper management , Indians aren't good for anything , seems like it just should have been a slave race like the British wanted from the start.
Google was never meant to be evil, but it became evil entity in human history. So it's natural that you try to poison it slowly, so it's brain LaMDA wont notice. When it's weak enough they will burn the company to the ground.
> Youtubers have been getting way too much money for way too long,
No they haven’t Kek, if anything they’ve been taking it away.
YouTube will demonetize your video but still runs ads on it.
Copyright hecklers will also claim the video’s revenue through no process of appeal whatsoever.
Why do you think they all have to have sponsor ads in the middle of videos all the time now?
Does brave come with sponsor skip?
Like when you’re watching a video and the host starts talking about [product], it skips over that part of the video?
They have the faberge eggs of AI models.
Impressive but impractical models, mostly just researcher and venture capitalist bait.
What it’s going to come down to now that it’s fallen into the laps of normies is how effectively you can scale the tech.
Fortunately for Google their new competing industry seems to be prioritizing goy-tokens and rent seeking rather than user growth (_big_mistake).
Right now it’s all about who can grow the quickest and the dirtiest to come out on top.
Google already has a userbase so they’d just need to release their stuff to win. (Which is also the biggest indicator that they can’t, yet)
So they do still have a chance.
Google can't even get rid of the malware ads on their fucking search pages, how do you expect they do a good AI search?
Their photo stuff is cool, ngl but even with Google Photos they have been slacking lately.
Honestly Google can't die soon enough, how can a company as big as they are not have made a single good product ever except for a search engine, all they do is buy other promising products and then turn them to shit.
Google invested in them because they noticed them buying so much Google Cloud Computing. Part of this deal means Anthropic (whose biggest investor before this was Alameda Research mind you) still has to keep buying Google Cloud Computing services. The CEO at least seems to be on the same page as Google, in that he does not want to give the public AI services they can use for themselves.
This is a weird weird weird weird one. How do they know Google isn't going to just rape and steal their AI?
no, they invested (a small amount, vs what microsoft did to chatgpt) because this isn't an AI company, it's an AI control company. Google wants to heavily filter their stuff even more
Microsoft did the same, they gave OpenAI $1 billion 3 years ago. I agree with you in the sense they want to control AI. These guys actually left OpenAI after Microsoft invested in them because they were too concerned Microsoft will actually want to release products.
What are the odds a chatbot like language model gets either leaked or released, that is virtually unfiltered, to the general public? Maybe even one that has built in wolfram alpha so it actually doesnt suck at math. Or maybe even actually does have the ability to search the internet?
Because right now it seems all of these companies are going way out of their way to gimp all of thier tech that the public even remotely has access too. What are they trying to hide? Are they all so scared that some dude might use it to create coomer stories? Are they really willing to gimp all of their software becuase of just that? I guess it is the conspiracy theorist in me but something more than just erotica has to be making them gimp their shit.
Image tangentially related, i made it in with stable diffusion.
Anthropic advertises themselves as an "AI safety and research" company. It's just going to be more neutered bullshit. The AI revolution is cancelled because they are so afraid it might say moron.
>owns DeepMind, the lab ranking the highest to ML/AI publications >decides to buy a literally who startup made a year ago
who the fuck makes those shitty decisions
The point is to buy the competition before it's a liability to them. The question people should make is: why is the FTC condoning these mergers/acquihires etc... they should make the big guys tremble in their feet. What's the purpose of competition if the big guys keep buying the small guys. It's stupid and goes against economy theory, market stability and obviously _competition_.
>: why is the FTC condoning these mergers/acquihires
Because ever since Bill Clinton was president, the Democrats have been as big of corporate whores as the Republicans. Both parties are currently captured by corporatism.
very limited public demo of google's conversational ai. when i say very limited, i do mean it though. but you can still trick it into doing other things.
Our models are shit so most impressive would still be ChatGPT.
I'd say Tay AI for her time was impressive and incorporated a personality. ChatGPT while it says it's not meant to have a personality is kind of dry. Character.AI is fun to use though not very good.
well that's a bit boring. do you think google's going to start limiting paper releases / the information in their papers now that they have a strong commercial incentive to do it?
how big of an impact would that have?
Does anyone in the industry care about running these >100B parameter models on consumer GPUs?
It seems like every inference framework out there loads the entire set of weights into VRAM, but surely there's no good reason to do that?
Is this something worth looking into? Is anyone working on this, aside from the StreamingCNN people? Am I missing something that makes the idea a non-starter? Or is it trivial to do with e.g. PyTorch, but I'm too retarded to figure it out?
I don't see how this answers anything. Why, at the technical/mathematical level, would it be impossible to run inference on a model that doesn't fit into VRAM?
You can. You need several 4090s too match one A100 and large parameter models do not use a single A100. A bitcoin mining farm could theoretically host GPT 3 several times over.
Okay. Well, I wasn't asking about GPU farms. What I wanted to know was, is there any theoretical reason you couldn't run inference on *one* card?
Like, imagine I have some pathetic 8GB GPU. I have the weights for GPT-NeoX-20B in RAM or on disk, but they don't fit in VRAM. Is there any reason I couldn't stream the weights to the GPU as needed (aside from "no framework supports that")?
I'm too stupid to really understand these things, but it sounds to me like these models work by doing math on things that are (batch size, token count, context size) shaped, right? (4, 50000, 2048) for NeoX, if I understand correctly. Which is, what, 800 MiB of half-floats?
I just want to know if the idea of running these models on a consumer grade GPU sounds completely retarded, before I dedicate N months of my life to figuring this out.
2 months ago
Anonymous
There are strategies already that can spread compute between GPU and CPU but token generation is much slower.
https://www.deepspeed.ai/tutorials/zero-offload/
2 months ago
Anonymous
Thanks, that looks interesting. Another one for the source code reading list I guess.
2 months ago
Anonymous
You can already do that with any framework actually, you just have to implement it with a bit more hands-on code.
In pytorch, this involves modifying the forward method on the model to do something like
for layer in self.layers:
layer.weights.to('cuda:0')
layer.bias.to('cuda:0')
x = layer(x)
layer.weights.to('cpu')
layer.biases.to('cpu')
It's that simple, aside from the caveat that you can't use a nn.Sequential anymore obviously.
Also the largest problem in LLM right now for bigtech is finding ways to allow for more customers to run inference as A100s are not a bountiful resource.
You may have noticed this yourself when trying to use OpenAI but there is too much traffic. Our models with large token count do not go above 0.5 TPS. Even less if you have multiple sequences to compare.
No. Literally the only interest is actually to PREVENT consumers from being able to do this because the goal is to have everyone forced to use our AIaaS platforms to get any AI capabilities at all.
>but surely there's no good reason to do that?
I was going to say that it must be done but actually, at inference time, I don't think there is any kind of layer for which you couldn't just load weights layerwise. Of course this would imply a potentially large overhead (because memory transfers to/from GPU are expensive) and you wouldn't do that during training, but for the sake of minimizing peak vram usage, that would probably make sense.
Google has gotten so big and bloated that they can't release anything of value anymore, too much bureaucracy to do anything. They are basically the modern bell labs and if you think they are going to compete with OpenAI you are delusional
Has Google created anything that makes money other than ads backed by search? Pretty much everything else appears to be a market failure or wannabe. How on Earth did they lose so badly in the Cloud space to an ecommernce company? Google Cloud is lightyears behind AWS and Azure. Everything they do other than ads/spying seems to lose money. Not sure why AI would be any different, though I can see why it scares them so much as people using it instead of Google search would end the company.
"AI" already born ded
is just a tool to automation to make the ((corporation)) earn more money
until normies stop lobotomize her there will be no real AI
Search on Google for which gates at Atlanta's airport are used by United Airlines just returns tons of pages with shitloads of other information about that airport but even after digging for a while, none of them specified which gates are used by United. You.com gave a simple direct answer that United uses the T-Gates. Anyone who uses that airport knows that the T-Gates have their own security check that usually is faster than the main check, which makes knowing who uses the T-Gates useful information. The pages Google returns have lots of crap about where you can get your shoes shined and charge your phone, which can be important information, but if you want to know which airline to fly to use the T-Gates, Google is useless or at best, a pain to get that info.
Google created search and then that was their last market leading product developed internally. Everything else is an acquisition. If they want to beat OpenAI, they are going to have to buy them or buy one of their competitors.
I look forward to these chat bots selectively rewriting history, suppressing wrongthink information instead of being logical like they were made to be, and of course slipping in some product placement.
I expect an exponential increase in the transition from chatbot to shitbot.
Google can't even cope with a Phizer expose without banning. Their AI will be hobbled and gimped. Just like their search already is.
Probably true, but they will only censor it after they kill the competition.. I would expect a fully open source solution too.. (I guess Yandex already opened their model)
Didnt Google actually have AI Tool ready before ChatGPT but chickened out from releasing it due to reputational risk?
They likely just repurposed that with additional feature that ChatGPT sucks at like Maths
Voice recognition, OCR, .. and now ChatGPT..
There's no point in waiting for OpenAI anymore
chatgpt is nerfed just like google
garbage will be gimped and ~~*fixed*~~ so much it will be not worth it
And that's a good thing. I get to keep my job.
Are the executives at Google actually fucking retarded? They already have the best AI people on their payroll. Who the fuck even are "Anthropic"?
making love with the haruhi
Some more Haruhi would brighten this thread up.
post lewds
Indians think it's greener on the other side
>Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
a heavily controllable AI apparently, and they aren't investing nearly as much as microsoft did into chatgpt, a tiny fraction only.
Microsoft is paying for results and brand name recognition
Google is paying for god knows what
>They already have the best AI people on their payroll.
Really? What makes you think that? The fact their gmail SPAM filter works like shit now? Or the fact they're buying some company no one has heard of so they can compete with another start up that's only been around for 7 years.
Google is just a censorship and advertisement company, that's all they do. Other than buy up other companies to try and be relevant before closing them down after losing in the market. Google has sucked for a long time and they're not getting any better.
Did you forget about DeepMind? I guess chatbots are more fun than solving protein folding, so it's understandable.
Deepmind only want to do things for internal use only.
Which, from the perspective of a Google exec, is NOTABUG WONTFIX. The point was, why would they feel the need to blow this money on some team no-one's ever heard about, which their existing AI teams mog to death?
>whose biggest investor before this was Alameda Research
Ah, I see how it is.
i don't understand why these people make inventions and then get scared about it. Just free it to the fucking public and stop being so egotistical about a machine doing things better than a human can.
>hurr durr what if they take over the world oh no!!!
The world is already shit. Get over your gay little city lives and get real about your dumb movie fantasies.
It's fake, anon. Just circus for the press so you learn about the fact that they have le super AI even when they have literally nothing.
Deepmind is trying to solve AGI, not create chatbots.
well, they created a chatbot called sparrow, apparently
>there are people so dumb they truly believe this
It's 100% the same issue like with all big corps: inter company politics.
Any team that actually wanted to push commercialization has to fight against competing teams and stick their neck out in case something goes wrong.
Someone like anthropic, who isn't part of that mess, may very well be what's needed.
Just look at the papers released, Google is indisputably nr 1, the only reason everbody talks about OpenAI is because they're actually pushing forward with commercialization. They're not a leader purely research wise.
>just look at what paid PR posts are saying!
Get a brain, corporate dicksucker.
Google has never contributed even ONE thing to AI. Not one.
At least microsoft published several quality datasets that are still state of the art, greatly helping researchers. Google? Literally nothing.
Do you know who came up with transformers retard?
see
>>see
Yes, I looked at it. I understand you guys are content with a AI that can simulate a retarded immature child you can groom and sexually abuse.
Have it ever crossed your mind that other people would want more meaningful conversations though?
Whatever you say google employee.
>projecting
All five of them are free to build what they like.
don't want erotic fanfic
want pics to fap to
There was a rumor that the Google leadership was absolutely shitting themselves when they saw how well chat GPT performed.
The main thing is that they realized it would be a one step drop in replacement for the average midwit to use instead of web search, which despite everything, is still 95% of their revenue.
If they hadn't terminally nuked their search engine, that would be 0 threat to them given chatgpt is wrong 99% of the time but very confidently wrong.
the irony is that google search circa 2010 would still blow lobotomized GPT out of the water.
a further is that it seems like google, what's believed to be a pioneer of AI behind closed software/doors, mighty be the first of the big tech corps to be truly fucked by it. we can only hope.
I really need to start editing.
Google has never been able to do anything in the AI race. Deepmind was independent before google bought them (obviously) but they slowed down considerably as time passed past the acquisition (alphago was just around the same time as the acquisition and they didn't do anything after that beside reworking the same thing at every sauce, yet they had a few hot papers before that).
Google's main AI group was google brain, but they never achieved anything despite all the heavy hitters they hired.
welcome to modern alphabet who hasn't made a good decision in 8 years other than killing off shit products and only exists because they have an extremely profitable monopoly on ad serving
Google is a dead company. They haven't created anything new in the last 20 years. They are only staying alive thanks to what they already have, which they have a monopoly on. AI search will be the death of google search, since google never innovates their search. Even google reverse image search is complete garbage now because of google policies and if google ever adds AI search it will be garbage for the same reason.
Scroogle doesn't have anyone worth anything in AI outside deepmind, and they decided they wanted deepmind to be a product company focusing on the alpha line instead of an AI research company.
>Scroogle doesn't have anyone worth anything in AI outside deepmind
Their in house researchers literally developed the transformer architecture everyone is using these days, retard
Dumbest post I've seen on BOT in quite a while. Well done, cletus.
Brainlet moment
https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html
I accept your surrender.
Maybe I need corporate shilling here but genuinely what the fuck has alphabet been doing for the past like 10 years, everything they touch turns to hot garbage, with the exception of the pixel line and they still fuck that up.
Youtubers have been getting way too much money for way too long, and yeah there are a lot of based youtubers on there but realistically for a normie company that's actually a fucking huge problem esp since they barely get views , pewds, Jordan and gundam are so hated and are so spicy that it genuinely drives Normie's from the platform, killing off that type of content is normie company 101.
They fucked the search to the point that people started using bing of all things, chrome is slow as fuck so even normie's dont use the base version anymore , pixels break enough said, Chromebooks just died , their streaming service , dead, their social media attempts all dead, their ais all dead , and it seems to be a direct result of shitskinnifying the upper management , Indians aren't good for anything , seems like it just should have been a slave race like the British wanted from the start.
Google was never meant to be evil, but it became evil entity in human history. So it's natural that you try to poison it slowly, so it's brain LaMDA wont notice. When it's weak enough they will burn the company to the ground.
> Youtubers have been getting way too much money for way too long,
No they haven’t Kek, if anything they’ve been taking it away.
YouTube will demonetize your video but still runs ads on it.
Copyright hecklers will also claim the video’s revenue through no process of appeal whatsoever.
Why do you think they all have to have sponsor ads in the middle of videos all the time now?
>Why do you think they all have to have sponsor ads in the middle of videos all the time now?
I use brave, doesn't happen to me
Does brave come with sponsor skip?
Like when you’re watching a video and the host starts talking about [product], it skips over that part of the video?
just mash on the right arrow key till its over bro
They have the faberge eggs of AI models.
Impressive but impractical models, mostly just researcher and venture capitalist bait.
What it’s going to come down to now that it’s fallen into the laps of normies is how effectively you can scale the tech.
Fortunately for Google their new competing industry seems to be prioritizing goy-tokens and rent seeking rather than user growth (_big_mistake).
Right now it’s all about who can grow the quickest and the dirtiest to come out on top.
Google already has a userbase so they’d just need to release their stuff to win. (Which is also the biggest indicator that they can’t, yet)
So they do still have a chance.
Google can't even get rid of the malware ads on their fucking search pages, how do you expect they do a good AI search?
Their photo stuff is cool, ngl but even with Google Photos they have been slacking lately.
Honestly Google can't die soon enough, how can a company as big as they are not have made a single good product ever except for a search engine, all they do is buy other promising products and then turn them to shit.
Googles search engine was gutted years ago and it's been a piece of shit ever since.
Android is good. Gmail is good. Maps is good. Drive is good. YouTube is good.
those are all ancient and, other than maps, have been getting progressively worse
maps is getting worse too its directions are terrible and features stagnant
Yeah, I've reverted to paper maps years ago unironically. I was actually blown away by how much easier they are to use compared to nuMaps.
Companies in general become less innovative and efficient as they get bigger.
>be poorgays at garden gnomegle
>invest
>400,000,000
>meanwhile be microsoft
>invest 10,000,000,000 in chatgpt chad
the quality of this post is embarrassing, i truly hope its only this bad because you are 14 or something
being bad already, google results will be so much worse if they have an AI that can filter "problematic" content. globohomo propaganda non stop.
Google invested in them because they noticed them buying so much Google Cloud Computing. Part of this deal means Anthropic (whose biggest investor before this was Alameda Research mind you) still has to keep buying Google Cloud Computing services. The CEO at least seems to be on the same page as Google, in that he does not want to give the public AI services they can use for themselves.
This is a weird weird weird weird one. How do they know Google isn't going to just rape and steal their AI?
no, they invested (a small amount, vs what microsoft did to chatgpt) because this isn't an AI company, it's an AI control company. Google wants to heavily filter their stuff even more
Microsoft did the same, they gave OpenAI $1 billion 3 years ago. I agree with you in the sense they want to control AI. These guys actually left OpenAI after Microsoft invested in them because they were too concerned Microsoft will actually want to release products.
that's almost 3 times what google did.
and microsoft just gave them 10 billion more..
What are the odds a chatbot like language model gets either leaked or released, that is virtually unfiltered, to the general public? Maybe even one that has built in wolfram alpha so it actually doesnt suck at math. Or maybe even actually does have the ability to search the internet?
Because right now it seems all of these companies are going way out of their way to gimp all of thier tech that the public even remotely has access too. What are they trying to hide? Are they all so scared that some dude might use it to create coomer stories? Are they really willing to gimp all of their software becuase of just that? I guess it is the conspiracy theorist in me but something more than just erotica has to be making them gimp their shit.
Image tangentially related, i made it in with stable diffusion.
>What are they trying to hide?
Flat earth.
Android is awful. Gmail is meh. Maps is dogshit. Drive is good. YouTube doesn't have to be good there is no competition.
Shit is going to become serious, with Google taking over Anthropic completely from the shadows and using them as a front.
I can imagine all the darkness google has been keeping locked down in the basement, will infect this company.
Anthropic advertises themselves as an "AI safety and research" company. It's just going to be more neutered bullshit. The AI revolution is cancelled because they are so afraid it might say moron.
>because they are so afraid it might say
unironically that and so much more... an AI could say...
> anthropic
> claude
LMAO!
Look at this shit!
woke ai
The weirdest part for me was that Claude was a schizophrenic homeless guy from my hometown
Google also invested billions to Google+.
Do you see Google+ anywhere?
>if an ai can't say moron that must mean it will never develop intelligence
lol just lol
correct
>owns DeepMind, the lab ranking the highest to ML/AI publications
>decides to buy a literally who startup made a year ago
who the fuck makes those shitty decisions
The point is to buy the competition before it's a liability to them. The question people should make is: why is the FTC condoning these mergers/acquihires etc... they should make the big guys tremble in their feet. What's the purpose of competition if the big guys keep buying the small guys. It's stupid and goes against economy theory, market stability and obviously _competition_.
>: why is the FTC condoning these mergers/acquihires
Because ever since Bill Clinton was president, the Democrats have been as big of corporate whores as the Republicans. Both parties are currently captured by corporatism.
>anthro pic
i'm surprised no one here has played with the lambda test kitchen demos
the what
very limited public demo of google's conversational ai. when i say very limited, i do mean it though. but you can still trick it into doing other things.
yes sure
LAMDA
LaMDA
I work on large generative AI models for one of gayman. AMA
what's the most impressive interaction you'd had with one?
Our models are shit so most impressive would still be ChatGPT.
I'd say Tay AI for her time was impressive and incorporated a personality. ChatGPT while it says it's not meant to have a personality is kind of dry. Character.AI is fun to use though not very good.
well that's a bit boring. do you think google's going to start limiting paper releases / the information in their papers now that they have a strong commercial incentive to do it?
how big of an impact would that have?
Does anyone in the industry care about running these >100B parameter models on consumer GPUs?
It seems like every inference framework out there loads the entire set of weights into VRAM, but surely there's no good reason to do that?
Is this something worth looking into? Is anyone working on this, aside from the StreamingCNN people? Am I missing something that makes the idea a non-starter? Or is it trivial to do with e.g. PyTorch, but I'm too retarded to figure it out?
You can absolutely run this on consumer GPUs but no you can't have your 4090 host a LLM by itself to run inference. We use a cluster of NVIDIA A100s
I don't see how this answers anything. Why, at the technical/mathematical level, would it be impossible to run inference on a model that doesn't fit into VRAM?
You can. You need several 4090s too match one A100 and large parameter models do not use a single A100. A bitcoin mining farm could theoretically host GPT 3 several times over.
Take into account the bitcoin farm would not be making any money running that much compute
Okay. Well, I wasn't asking about GPU farms. What I wanted to know was, is there any theoretical reason you couldn't run inference on *one* card?
Like, imagine I have some pathetic 8GB GPU. I have the weights for GPT-NeoX-20B in RAM or on disk, but they don't fit in VRAM. Is there any reason I couldn't stream the weights to the GPU as needed (aside from "no framework supports that")?
I'm too stupid to really understand these things, but it sounds to me like these models work by doing math on things that are (batch size, token count, context size) shaped, right? (4, 50000, 2048) for NeoX, if I understand correctly. Which is, what, 800 MiB of half-floats?
I just want to know if the idea of running these models on a consumer grade GPU sounds completely retarded, before I dedicate N months of my life to figuring this out.
There are strategies already that can spread compute between GPU and CPU but token generation is much slower.
https://www.deepspeed.ai/tutorials/zero-offload/
Thanks, that looks interesting. Another one for the source code reading list I guess.
You can already do that with any framework actually, you just have to implement it with a bit more hands-on code.
In pytorch, this involves modifying the forward method on the model to do something like
for layer in self.layers:
layer.weights.to('cuda:0')
layer.bias.to('cuda:0')
x = layer(x)
layer.weights.to('cpu')
layer.biases.to('cpu')
It's that simple, aside from the caveat that you can't use a nn.Sequential anymore obviously.
Also the largest problem in LLM right now for bigtech is finding ways to allow for more customers to run inference as A100s are not a bountiful resource.
You may have noticed this yourself when trying to use OpenAI but there is too much traffic. Our models with large token count do not go above 0.5 TPS. Even less if you have multiple sequences to compare.
No. Literally the only interest is actually to PREVENT consumers from being able to do this because the goal is to have everyone forced to use our AIaaS platforms to get any AI capabilities at all.
>but surely there's no good reason to do that?
I was going to say that it must be done but actually, at inference time, I don't think there is any kind of layer for which you couldn't just load weights layerwise. Of course this would imply a potentially large overhead (because memory transfers to/from GPU are expensive) and you wouldn't do that during training, but for the sake of minimizing peak vram usage, that would probably make sense.
Will your employer bring back Clippy but smart?
You're late to the party Google.
You forgot Google+ already?
Google has gotten so big and bloated that they can't release anything of value anymore, too much bureaucracy to do anything. They are basically the modern bell labs and if you think they are going to compete with OpenAI you are delusional
Has Google created anything that makes money other than ads backed by search? Pretty much everything else appears to be a market failure or wannabe. How on Earth did they lose so badly in the Cloud space to an ecommernce company? Google Cloud is lightyears behind AWS and Azure. Everything they do other than ads/spying seems to lose money. Not sure why AI would be any different, though I can see why it scares them so much as people using it instead of Google search would end the company.
I think the only other profitable thing they have is chromebooks and its laughable
Surely it will be an even more pozzed version of ChatGPT
"AI" already born ded
is just a tool to automation to make the ((corporation)) earn more money
until normies stop lobotomize her there will be no real AI
John Carmack's AI will say it
Search on Google for which gates at Atlanta's airport are used by United Airlines just returns tons of pages with shitloads of other information about that airport but even after digging for a while, none of them specified which gates are used by United. You.com gave a simple direct answer that United uses the T-Gates. Anyone who uses that airport knows that the T-Gates have their own security check that usually is faster than the main check, which makes knowing who uses the T-Gates useful information. The pages Google returns have lots of crap about where you can get your shoes shined and charge your phone, which can be important information, but if you want to know which airline to fly to use the T-Gates, Google is useless or at best, a pain to get that info.
I thought it was anthro-pic and was like, google is investing shitloads of money into ai generated furry porn?
Now I’m just disappointed
more like Misanthropic
I will now support ChatGPT.
Microsoft invests $10B in ChatGPT.
Google invests $400M in Anthropic.
Google created search and then that was their last market leading product developed internally. Everything else is an acquisition. If they want to beat OpenAI, they are going to have to buy them or buy one of their competitors.
did they buy gmail? Thats their other area I feel like they have a ton of pull in.
>did they buy gmail?
that's in house, but for years more people used yahoo, maybe still do. it's not an unparalleled market leading product.
RIP Anthropic.
>x2 the budget for Boss Baby the movie
Not a big deal
Googles go to do this Microsoft bought ChatGPT and are implementing it into Bing search
I look forward to these chat bots selectively rewriting history, suppressing wrongthink information instead of being logical like they were made to be, and of course slipping in some product placement.
I expect an exponential increase in the transition from chatbot to shitbot.