WiIl GPT improve? Like very soon?

WiIl GPT improve? Like very soon?

Currently GPT4 cannot even generate over 200 lines of code before it starts seriously fricking up. It also forgets things after each query - it drops the previous edits you asked for.
I'm lazy and just want to do 3d models for my vidya.
I don't want to code it.

ChatGPT Wizard Shirt $21.68

Beware Cat Shirt $21.68

ChatGPT Wizard Shirt $21.68

  1. 12 months ago
    Anonymous

    soom

  2. 12 months ago
    Anonymous

    The difference between gpt4 and gpt3 was less than the difference between 3 and 2. 5 will be less than that.
    The gays claiming otherwise are blatantly lying for an unknown reason and thinking that people can't clearly see what's happening

  3. 12 months ago
    Anonymous

    maybe they artificially limited it to save on running costs because so many people were using it

  4. 12 months ago
    Anonymous

    boop

  5. 12 months ago
    Anonymous

    computing requirements scale exponentially in relation to the token buffer size.

    • 12 months ago
      Anonymous

      wait is this true? if so we might still be writing code 50 years from now lmao

      • 12 months ago
        Anonymous

        If you want a lookback of n tokens the system needs a tensor of n^2 values internally. To go from n to n+1 tokens you'll be processing an additional 2n+1 values, with at least (n+1)^2 calculations being performed rather than n^2. 200 lines of code with an average of around say 5 tokens per line is about 1000 tokens. To generate more working code without problems you'd need a larger token buffer as old tokens will be dropping out of the buffer and context for the generated text will be being lost.

        This is all just speculation of course based on my understanding of how such systems work.

        • 12 months ago
          Anonymous

          nice
          I don't think it's very useful to continually increase the context tokens as much as we'd need a sort of management for it. Like long term memory so it offloads least priority context, and can recall it from storage according to certain criteria.

        • 12 months ago
          Anonymous

          So we might race towards fast clock and parallelism?

          • 12 months ago
            Anonymous

            The tensor must be processed in its entirety for each token that is generated. We're already doing fast clocks and parallelism with GPUs, but the data still needs to be sent over a data bus and collated somehow. It seems that signalling generates a fair amount of heat.

            The recent controversy around using mmap in llama.ccp is somewhat related. Sure you don't need the whole model in memory all at once but the virtual memory thrashing that results from all of the model weights being needed to contribute to the generated output text tokens makes the process very slow. It's a trade of space for speed.

        • 12 months ago
          Anonymous

          Nice get. Could something like "context layers" resolve this. Perhaps have a "compressor" that turns earlier tokens into general statements based on importance, so that the short-term memory of an AI doesn't get overloaded. Basically

          I'm not sure you can equate it like that. We can remember a lot of things (concepts, images), from storage, but our attention span is not so large. We can focus on a few things at once. We'd have a good time with something that can be just a bit better than that.
          How much would the attention span of a human equate in tokens?

          So we might race towards fast clock and parallelism?

          I don't think that parallelizing an AI would help if every intelligence upgrade requires 2^n servers. You'd have to discover a new algorithm every time. Parallelizing is better for speed and lower hardware requirements.

          • 12 months ago
            Anonymous

            Parallelizing has its own associated communication costs right? Isn't this what chiplets are seeking to mitigate?

            • 12 months ago
              Anonymous

              Yes. The point of parallelization right now is to spread a big model across multiple inexpensive servers instead of one server with ridiculously high specs. Chiplets accomplish the same idea, but are packed closer together for speed.

              • 12 months ago
                Anonymous

                Sure. But how will the creators of AI systems know which relationships are important to keep locally and which aren't?

              • 12 months ago
                Anonymous

                I don't know.

              • 12 months ago
                Anonymous

                I don't think anyone does know at this point. That's the crux of the matter. Better algorithms need to be developed it seems to me.

              • 12 months ago
                Anonymous

                AGI will fix everything :^)

        • 12 months ago
          Anonymous

          >5 tokens per line
          You're being generous

          • 12 months ago
            Anonymous

            I did say on average. Some lines might be a lot more than 5 tokens, but others might just be "}"

    • 12 months ago
      Anonymous

      This frustrates me as well. I try to give it bullet point feature lists for classes, and gpt4 generally does an amazing job, but it times out and stops working beyond the 200 line mark. It's made me try to keep my classes short. It definetely speeds everything up 10x. If you have clear specs for classes it will write them well.

      If there is a scaling issue moving forward, as in gpt5 can't scope up beyond the 200 line mark, or get a contextual feel of the whole project, I don't mind. It's invaluable as it is already.

      What would be nice to see is if gpt could factor in the whole structure of the project into the writing of code, so it could operate architectually, but if is true it might be further out than I'd hope.

      I don't think piping in different kinds of data (video and images) will help gpt5 get better at this kind of thing as well. I could be wrong.

      • 12 months ago
        Anonymous

        computing requirements scale exponentially in relation to the token buffer size.

        I don't know shit about how this all works but from what I do understand, the bigger picture, technology itself is getting exponentially more advanced.
        However many 'tokens' are needed for an AI to understand an entire project shouldn't matter if we're going to reach AGI within 2023, right?

        • 12 months ago
          Anonymous

          I suppose it comes down to whether you believe Moore's law can continue to operate or or not.

          • 12 months ago
            Anonymous

            Well, according to Google...

            • 12 months ago
              Anonymous

              Some people would become quite cross if it were I'd imagine.

              How many numbers in a phone number (tokens) can humans remember? 7 at most? It's my opinion brute forcing AGI isn't the right approach.

              • 12 months ago
                Anonymous

                I'm not sure you can equate it like that. We can remember a lot of things (concepts, images), from storage, but our attention span is not so large. We can focus on a few things at once. We'd have a good time with something that can be just a bit better than that.
                How much would the attention span of a human equate in tokens?

              • 12 months ago
                Anonymous

                Uuhhh I think you can't really tell since it's all in our heads, you can't measure perception since we haven't even got a technical language for that, we can't even quantify awareness except from people's expressions.
                Humans brain has the structure it has because we have got limbs that extend our nervers, we should replicate neuron on a chip and make a true neural net to shit ourselves and eat shit and piss and die in cum dumpster genitals.
                Happy 3aster goy.

              • 12 months ago
                Anonymous

                The human brain seems to be like a whole federation of smaller, special purpose neural networks right? One just to recognise faces for example.

              • 12 months ago
                Anonymous

                Neurologists know this shit, or have good insights.
                Now LLMs might be something that we have to mimic closer to how we work but no need for the say math part of the brain. I think Wolfram is way better. No need to replicate how we do math in our brains. We need to replicate enough that will allow us to easily interface to it. Other things we might want to upgrade as it were.

            • 12 months ago
              Anonymous

              the amount of people that think moores law is dead doubles every 18 months

    • 12 months ago
      Anonymous

      It scales quadratically, but not with Hyena which is n log n

      • 12 months ago
        Anonymous

        I'm no mathematician. But even I can see this scaling won't be able to go on for long and each new iteration is only bringing marginal improvements along with a massive increase in costs. It just doesn't seem a good strategy economically IMHO.

        Of course people buy snake oil all the time and this trend is troublesome in its own right. It's wish fulfilment for CEOs.

    • 12 months ago
      Anonymous

      I am too brainlet to understand half this thread. So I asked GTP-4, but his info is not up to date.

      Are these methods viable?

      Memory-efficient architectures
      Model distillation
      Adaptive computation
      Incremental learning
      Hierarchical models
      Sparse attention mechanisms
      Dynamic model pruning
      Low-rank approximation
      Weight quantization
      Parallel and distributed processing

      • 12 months ago
        Anonymous

        You're like a waiter, you serve food but you have nothing to do with the delicious food being served. You are easily replaceable and will have little merit, easily forgotten unlike the chef and his glorious dish.

        • 12 months ago
          Anonymous

          I am more like the customer getting the food, anon. Spoonfed and all. I am here to enjoy the work of others with as little effort as possible :3

          • 12 months ago
            Anonymous

            I stand corrected, you are worse than a waiter. You are delusional enough that you believe you are the person being served versus the reality that you're trying to interject into discourse that you add no value to. Well done, just like the dish.

            • 12 months ago
              Anonymous

              It's always inherently humorous to me when I trigger the autism of anons like you. I inquired a question out of curiosity, even went as far as to clarify that I am not educated in the subject, its actually quite new to me. At no point was I looking to add value to a topic I admitted not to know about, which just by asking i reckon is enough to imply it, but seems like that wasnt apparent enough. I don't work with this, I hold no personal interest beyond fleeting curiosity caused by the initial post. I am learning new things and felt like asking about it as OP seemed to be somewhat informative.

              Somehow this seemed to offend you, which admittedly is quite entertaining to see kek.

              • 12 months ago
                Anonymous

                BOT hosts bottom-of-the-bucket individuals

              • 12 months ago
                Anonymous

                Lack of self-awareness I think. Also the expression is bottom-of-the-barrel.

                People take things more seriously than needed, perhaps because how specific the topics are as they are populated with two purposes, the discussion of interests merged with the anonymous culture of BOT. The later tends to make anons forget that this is essentially just a social media platform at the end of the day, little more than entertainment.

  6. 12 months ago
    Anonymous

    uh oh STINKYYY

  7. 12 months ago
    Anonymous

    No, GPT is a glorified copy paste machine that simply idiotically mimics any dumb shit it got of the internet. "Generative" AI should be called "Parrot" AI

  8. 12 months ago
    Anonymous

    >WiIl GPT improve?
    Yeah, in the same way that other technologies improve. That's why our cars are flying now.

    • 12 months ago
      Anonymous

      And self-driving, a feature Memelon has been promising every year for 6 years

      • 12 months ago
        Anonymous

        More than 10 years now for the promise of full self driving isn't it?.

        • 12 months ago
          Anonymous

          Yeah, but I mean 6 years as in 6 years straight of Elon coming out publicly and saying "Next year we'll have self-driving cars" for 6 straight years every year he's been saying that

          • 12 months ago
            Anonymous

            I thought he started doing that in 2013, but then again I haven't been paying attention. Maybe GPT-5 can be made to drive cars if it's AGI.

    • 12 months ago
      Anonymous

      Based numberinos of the slot machine 666 allah is pork mossad vs mohamed

  9. 12 months ago
    Anonymous

    >WiIl GPT improve? Like very soon?
    Maybe, but probably not. If they use the same tech they've done previously, training the next step up will require 4 or 5 months of dedicated time on a supercomputer somewhere round 100x larger than the current world's largest. It's insanely difficult to keep things stable at that scale as you have to deal with crazy things like the natural radioactivity of the atmosphere.
    Changing technology to something that scales better is possible, but will take multiple years to do as most investment has been in massive GPUs; the different tech will be not at all the same. (I happen to think that neuromorphic tech has good potential, provided it is standard digital. Exotic stuff with analog computing and memristors ad shit like that is academically interesting, but not production-ready.)

    • 12 months ago
      Anonymous
      • 12 months ago
        Anonymous

        Dublin down.
        The next system needs to be about two orders of magnitude larger than the current largest, and then used for far longer on one task than the current largest is ever used. Building it is going to take literal billions and stop a lot of other uses getting anything. This isn't about putting a bigger GPU in. This is about putting half a million of those bigger GPUs in.

  10. 12 months ago
    Anonymous

    I'm so used to GPT4's perfect English that human English sounds like incoherent garbage.

  11. 12 months ago
    Anonymous

    api fixes all that, if you want to use it in a serious way, then get the API and build a serious tool

  12. 12 months ago
    Anonymous

    >

    Currently GPT4 cannot even generate over 200 lines of code before it starts seriously fricking up. It also forgets things after each query - it drops the previous edits you asked for.

    We told you this the whole time, but you fell for the "AGI is here we're all gonna die" hysteria

    • 12 months ago
      Anonymous

      Maybe GPT-5 can do 400 lines before fricking up lol

  13. 12 months ago
    Anonymous

    >just worked with gpt4 to pull a list of all our shipments for the day, categorize them by store, and email it to my boss each day
    Bros, I'm basically a programmer now

Leave a Reply to Anonymous Cancel reply

Your email address will not be published. Required fields are marked *