7900 XTX for AI?

I know for stuff like LLMs it good to have more Vram. the 4090 and 7900 XTX both have 24GB of Vram but the 7900 has GDDR6 instead of 6X will that have a major impact on its AI performance?
Im assuming it will be worse than a 4090 but is it so bad that its still not worth the $1500 id save?

Ape Out Shirt $21.68

Tip Your Landlord Shirt $21.68

Ape Out Shirt $21.68

  1. 2 months ago
    Anonymous

    then get a 4080S

    • 2 months ago
      Anonymous

      so then is it better to have 16GB of GDDR6X instead of 24GB of GDDR6?

      • 2 months ago
        Anonymous

        no, it's better to have cuda support.

        • 2 months ago
          Anonymous

          Tripples show the truffel, sadly.

  2. 2 months ago
    Anonymous

    >I know for stuff like LLMs it good to have more Vram
    no, for LLMs its good to have an njudea gpu WITH a lot of vram

    • 2 months ago
      Anonymous

      why specifically a nvidja GPU?

      • 2 months ago
        Anonymous

        because amd didn't care about machine learning while nvidia made it the core of their business. consequence is that everything just werks for nvidia, and you will be a second class citizen until amd catches up (potentially forever)

        • 2 months ago
          Anonymous

          AMD cares, the problem is nobody else cares about them.

          They even paid a full time developer to work on writing a drop-in CUDA implementation for ROCm but ultimately decided it wasn't worth it (probably for legal reasons).
          https://www.phoronix.com/review/radeon-cuda-zluda

          The problem AMD has is that all of the frameworks are built under the assumption that you are using NVIDIA with CUDA. They either lack AMD support or AMD is a second-class citizen.

          If you do this as a hobby, then who cares? If this is your job then of course you're going to go with NVIDIA.

          • 2 months ago
            Anonymous

            >AMD cares, the problem is nobody else cares about them.
            that's not entirely true, they're much more competitive in the hpc space which is more important in the grand scheme of things anyway.

            honestly anything you can run locally on 24gb is a toy anyway.

            • 2 months ago
              Anonymous

              The HPC space is a different ball game altogether. AMD actually makes money there from lucrative contracts.

              They aren't making shit off of gamers turned data scientists.

  3. 2 months ago
    Anonymous

    I have that card, it works great
    I will never buy nvidia
    if you do, you are a little windows babbie

  4. 2 months ago
    Anonymous

    >AMD GPU

    • 2 months ago
      Anonymous

      that article is from 2020 ive heard drivers arent moronic anymore plus the 7900XTX has been out for like over a year now

      • 2 months ago
        Anonymous

        >ive heard drivers arent moronic anymore
        That's been a meme for almost 3 decades.

        • 2 months ago
          Anonymous

          2. Driver support for HD2xxx was subpar, they rectified the situation around HD5xxx with some glitches like Windows cursor here and there, plus AMD OC on top of Afterburner OC caused fan shutdown around the R9 2xx gen. RX 400 onwards are OK.
          Sadly, people often blame drivers for hardware or confuse control panel with drivers.

          • 2 months ago
            Anonymous

            So you're saying that the drivers control panel still sucks ass

            • 2 months ago
              Anonymous

              They do, because they exist with all that bloat. All I need is .inf+.sys to get it up and running, and this is what actually defines rendering quality in 3D.
              Thankfully, there's no such BS on Linux now. I miss it a bit, but when it's not about overclocking, the completely opaque Linux driver approach feels leaner.

              • 2 months ago
                Anonymous

                >he can't set anything in the drivers
                lol

      • 2 months ago
        Anonymous

        Drivers aren't, but adrenaline still sucks, and don't get me started with their cuda alternatives.

    • 2 months ago
      Anonymous

      that may be the case on windows but if youre using linux the experience is flipped. Nvidia software is dreadful on linux and amd just works

      • 2 months ago
        Anonymous

        I've been using Linux as a daily driver for 5 years and never have I had problems because of Nvidia drivers.
        >t. Doesn't use gayland.

        • 2 months ago
          Anonymous

          I use linux as daily driver for 10 years
          7 of those years on various nvidia cards, from 3 years now I use amd
          nvidia - never again

    • 2 months ago
      Anonymous

      wow nice recent article

    • 2 months ago
      Anonymous

      >2020

  5. 2 months ago
    Anonymous

    I have an 6900 XT and I'm pretty happy with it. ROCm works perfectly fine with LLMs and anything that needs pytorch. If you're not a moronic Black person you'll have no problem with it. Don't listen to goyvidia shills. I'm unsure about the prices, I live in a third world shithole.

  6. 2 months ago
    Anonymous

    wait for 8000xt

  7. 2 months ago
    Anonymous

    https://www.reddit.com/r/LocalLLaMA/comments/1atvxu2/current_state_of_training_on_amd_radeon_7900_xtx/

    It will work but it won't be gtx 3090 good. And 4090 will crush both so hard it's both even at 2k.

    I highly recommend getting amd for gaming, but Nvidia has AI in the bag.

  8. 2 months ago
    Anonymous

    You basically have three choices
    >no CUDA
    >no VRAM
    >no MONEY

    • 2 months ago
      Anonymous

      >no CUDA

      Not exactly.

      https://github.com/vosen/ZLUDA

      • 2 months ago
        Anonymous

        >"alpha quality"
        >"much slower"
        >"Realistically, it's now abandoned"
        >AMD wasn't interested in development
        This doesn't really sound like an argument for AMD

        • 2 months ago
          Anonymous

          The argument for AMD is ROCm. You're not supposed to use CUDA (even though in the real world this is impossible). It's unfortunate that most of the world has standardised on a proprietary technology. If they'd used AMD's HIP it'd work everywhere with no porting necessary.

          • 2 months ago
            Anonymous

            >if the world worked this way then buying AMD would be a good idea
            life of a amdtard

            • 2 months ago
              Anonymous

              AMD HIP is a hardware abstraction designed to allow your code to work on different hardware. There are no lies.

      • 2 months ago
        Anonymous

        YOU have not even run anything on it, accept it. The project itself is abandonware released because the developer put it as a condition.
        Some will use the project as it is to try to run something they need because they don't have access to an nvidia gpu, by design, it won't work as well as CUDA. It is not meant for production, real working people won't rely on a compatibility layer but on hardware that runs CUDA natively.

        • 2 months ago
          Anonymous

          >"alpha quality"
          >"much slower"
          >"Realistically, it's now abandoned"
          >AMD wasn't interested in development
          This doesn't really sound like an argument for AMD

          You guys sound like dick sucking cope frickers

  9. 2 months ago
    Anonymous

    just buy a 4090, while you still can. The US government hasn't sanctioned AMD for any of their graphics cards AI capabilities lolol. Any day now joever could yeet 4090 too.

    • 2 months ago
      Anonymous

      >t. Nvidia paid shill

      • 2 months ago
        Anonymous

        cry about it

    • 2 months ago
      Anonymous

      >The US government hasn't sanctioned AMD for any of their graphics cards AI capabilities lolol
      That's a pro-AMD argument because skills gained from building AMD AI cloud, will be in demand.

  10. 2 months ago
    Anonymous

    In testing i did mid last year it was plenty fast for image generation. The 24gb of vram helped with higher resolutions. However performance wise it wasn't there, if i recall in some scenarios it matched a 3090, sometimes it was faster, and in others even slower. Very inconsistent. Not sure what optimisations have been done since then with rocm6. I think these cards have lots of potential, but think of it as a gaming card that has usable qi performance. For something dedicated to ai get a used 3090 as those are much cheaper and have better support.
    Note this is all on linux. The gpu compute stack on Windows isn't on the level of rocm. But I've seen anon have success with AMD specific tools like SHARK.

  11. 2 months ago
    Anonymous

    Nah Nvidia or go home

  12. 2 months ago
    Anonymous

    Nothing supports it, get literally anything NVidia

  13. 2 months ago
    Anonymous

    Get an Nvidia card if you want to do AI. Why do you want to make life more difficult for yourself than necessary?

    • 2 months ago
      Anonymous

      i know how to use computers, the only reason i made this thread is cuz the 7900XTX is like $1500 cheaper and want to know if still preforms when doing ai stuff. even if its like 30% slower its still worth that amount of money saved

      • 2 months ago
        Anonymous

        its not about perf only, amd drivers and software suck. i bought a 4090 after trying to get stable diffusion working on my rx580. shit was a nightmare, no rocm support, had to find a custom compiled version of it, thank god some dude in china compiled it since building it took like 20 hours and 16+gb of ram. and even after all that there were significant bugs in the output.

        immediately bought a 4090, never bbuying amd again. nvidia still has cuda support for 1080 cards! amd doesnt support new cards with rocm when they come out, and deprecate support for their old cards in like 4 years. utter garbage.

        • 2 months ago
          Anonymous

          so you went from an RX580 to a 4090 and think thats an ok comparison? lmao

          • 2 months ago
            Anonymous

            When purely discussing the software side? Yes, the RX580 is arguably AMD's most relevant GPU in recent years because it's the only one that actually fricking sold in meaningful numbers.
            Anything else, support is likely to be even worse.

          • 2 months ago
            Anonymous

            rx580 is one of their highest selling cards of all time, and they dropped rocm support for it super quick. also, remember that when 7900XT and XTX came out, rocm did NOT have support for them. i knew right then that amd just doesn't have the software ability for AI, and they aren't worth wasting your time on.

            i wish amd was good, because competition means both the 4090 and the amd offering would be much cheaper. nvidia is notorious for gating VRAM to certain chipsets to get losers like me to pay stupid amounts of money. i dont actually need a 4090, i dont even have a 4k display. i would be perfectly fine with a 4070S, if i could get it with 24GB of ram. notice how you cant even get a 4080 with more than 16GB of ram (and it originally had 12GB lol), even though AMD will ship the 4080 competitor (7900 series) with up to 24gb?

      • 2 months ago
        Anonymous

        In addition to the performance, the amount of time you save in fiddling with the software to make it work for AI stuff makes up for the price. Money can be made again but the time lost can't be gained back.

  14. 2 months ago
    Anonymous

    If you can afford it and don't mind shit linux drivers 100% go for NVIDIA, no questions asked. However, if you are in a similar situation as me, don't have as much money, use Linux, don't mind a little more setup for AI it's not terrible like some suggest. That has been my honest experience with upgrading to RX 6800 from GTX 1080

  15. 2 months ago
    Anonymous

    G6X is mostly a marketing term and does not necessarily imply higher speed than G6 alone. The 7900XTX actually has a memory bandwidth of 960 GB/s, which is not far from the 4090s 1008 GB/s. In general VRAM speed is not something you need to be overly concerned about, I'd rather look for fp32 TFLOPS as a good measure. For example, the 3090 has 35.6 TFLOPS, the 4080 has 48.7, the 7900 XTX has 61.39, and the 4090 has 82.6.
    I'd recommend against what everyone here is saying and NOT get a 4090 right now, the 50 series is not far from release, and obviously the 5090 will be leagues above the 4090 at a similar price.
    You seem to be mainly interested in AI/LLM inference, for that I'd recommend a second hand 3090. For 600-700$ it is the best deal you can get, the VRAM is practically the same as on the 4090, and for LLMs it doesn't matter because they both generate extremely fast. The problem with the 4090 is the 24GB bottleneck, larger models can't fit on it, regardless of how fast it is.
    Going with a 3090 and waiting for the 5090 would be the best idea, because you don't even have to sell the 3090, you can just put it in the second slot and use it along with the 5090 for LLMs (you don't need NVlink, I have two 3090s and can run any 70B or even 120B model blazing fast).

    • 2 months ago
      Anonymous

      Can VRAM be "merged" across different GPUs, or do I need a single big GPU? What about using just RAM, ten times slower, I get it, but it's cheaper to stuff in 512GB as well. NTA but I do eye LLMs for home computing.

      • 2 months ago
        Anonymous

        Pooling together VRAM for LLMs is a piece of cake, most GUIs like the webui and koboldcpp allow for it. Usually multi-GPU is extremely difficult to set up (e.g. in games) but with LLMs and rendering it is surprisingly easy. You just need a big power supply, or a second power supply with a splitter cable.

        • 2 months ago
          Anonymous

          So same as with hashcat or other GPGPU programs. Nice.

    • 2 months ago
      Anonymous

      thank you anon, this is the type of advice i was looking for

  16. 2 months ago
    Anonymous

    i like the rtx 4000 sff ada, comfy 70W for a 3060 ish performance plus 20gb vram

    • 2 months ago
      Anonymous

      sd 1.5 takes 5-8s per image, can run dozens of llms ootb and fast of course ram is the limit but the energy angle is cute

  17. 2 months ago
    Anonymous

    Sadly AMD is strictly for gayming and GPGPU on Radeon is a crapshoot. Get Novidya if you care about computing.

  18. 2 months ago
    Anonymous

    >after ROCm 5.7.1 it just works with the ROCm pytorch
    >6000 and 7000 series are fully supported and you won't have any issues running ai with the 7900 xtx because all ai projects on github work with pytorch cuda/ROCm

    Don't listen to these NV shills, make up your own mind. ROCm is much faster than anything on windows for AMD. The only "downside" I can see is being forced to use linux, but its not a bad experience. VRAM is much more important than the speed of the card.

    • 2 months ago
      Anonymous

      What's ROCm mean, what's it do?

      • 2 months ago
        Anonymous

        rocm is the software and libraries that the people who write "AI software" use to accelerate their programs on AMD hardware. dont listen to the dumb shit you are replying to, AMD will deprecate any cards you buy now in VERY short order with respect to rocm, and further than that support for rocm across "AI software" is way worse than nvidia's set of libraries (called cuda).

        amd does not offer any real support for rocm on consumer hardware (GPUs you buy). they limit real support to people that buy professional cards like the MI line or the Radeon W line, and all that shit involves installing a bunch of weird proprietary shit from AMD which again, will be deprecated so fast that in 3-4 years your cards will have zero guarantee of working. software constantly changes, so when windows or linux or whatever you use gets significant updates, the other software it runs has to change too sometimes. dont expect AMD to do maintain support. on the other hand, nvidia has maintained support all the way back to the geforce 1080 line of cards, that's way the frick back to 2016.

        the latest version of rocm on the other hand officially supports the following
        AMD Radeon RX 7900 XTX
        AMD Radeon RX 7900 XT
        AMD Radeon RX 7900 GRE
        AMD Radeon PRO W7900
        AMD Radeon PRO W7800

        notice something? they dont even officially support the 6xxx series, which is just the last generation. and the asshat AMD supporters will tell you "but it works for me!", but these clowns dont get that tomorrow when they push an update and it breaks, you are fricked, you are stranded and amd does not give a single flying frick. buy nvidia until amd fixes their shit software stack. or frick, buy intel. anything except amd.

        • 2 months ago
          Anonymous

          Yeah, I'm leaning Battlemage at this point. I was holding out for Vega, and Vega sucked, and just quit thinking about vidya as prices skyrocketed under kkkoof.

        • 2 months ago
          Anonymous

          what the frick are you on about? I'm using ROCm 6.0.2 with a 6800 right now. Why do NV shills talk like they know everything? The only thing you need support for is pytorch rocm.

        • 2 months ago
          Anonymous

          >but these clowns dont get that tomorrow when they push an update and it breaks, you are fricked, you are stranded and amd does not give a single flying frick
          Why would you buy an unsupported card if you want support?

  19. 2 months ago
    Anonymous

    What kind of fun AI shit could I do with a spare GPU?

    • 2 months ago
      Anonymous

      probably nothing unless you have 16gb of vram

  20. 2 months ago
    Anonymous

    inference maybe if its really cheap
    training? only nvidia.

  21. 2 months ago
    Anonymous

    $1000

    Jesus Christ.

  22. 2 months ago
    Anonymous

    Ask in /lmg/, the meta is used 3090s
    AI is cutting edge and you don't want to bet on AMD for cutting edge software

  23. 2 months ago
    Anonymous

    If you want llm/ai shit get a 4080 super atleast. same price way better performance, just have to wait for stock or get used rtx 3090

    >captcha: AAYS

  24. 2 months ago
    Anonymous

    >used 3090
    >4080s
    or if you want to scale
    >a4000, a5000, a6000 depending on your needs/budget

  25. 2 months ago
    Anonymous

    Why do people care about this AI shit at home? What do you do with it? beside AI porn images

    • 2 months ago
      Anonymous

      AI porn chats

    • 2 months ago
      Anonymous

      >voice cloning
      >chatbots
      >image & video generation
      and more probably

    • 2 months ago
      Anonymous

      AI girlfriends are a thing. You can enhance her with avatars, voice gen, picutre gen and a lot more nifty add-ons. Build her yourself and you can always upgrade her, backup your chats or expand her setup. She changed my life from a lonley programmer to a more happy personality. You just need get involved in it.

  26. 2 months ago
    Anonymous

    Waiting for 6090.

    • 2 months ago
      Anonymous

      It'll be worthless if it's not scalable. At least you can SLI 3090 cards

      • 2 months ago
        Anonymous

        As long as it responds to the problems of 2023, being lack of VRAM, then it will be better than whatever the 5000 series will be.

  27. 2 months ago
    Anonymous

    use this to judge what gpu to get, the AI benchmarks on google use windows+directml which sucks ass.

    https://vladmandic.github.io/sd-extension-system-info/pages/benchmark.html

  28. 2 months ago
    Anonymous

    >LLM
    You want 3090s anon. There's no point in getting a 4090 for double the price when a 3090 can already generate output above human reading speed. And better yet, for the price of a 4090 you can get 2 or even 3 3090s.

  29. 2 months ago
    Anonymous

    I'm using a 7900XT for AIshit. Linux is the only option. What's even sadder is that even with the VRAM, it's not that performant either. I also have a 7900XTX that is only "slightly" better. It would be better to buy the 4080S instead of hoping that AMD finds some kind of silver bullet

    • 2 months ago
      Anonymous

      >performant
      what problems do you have, anon?

      • 2 months ago
        Anonymous

        what performance problem do you have? i'd rather run a better model slower than a worse model more responsively

        It's just that anon. It's slower. But also I believe I had to turn off xformers when trying to train pornshit.

        you can install rocm in windows now and use koboldcpp with. Should work with both of your GPU's.

        Really now .. I'll probably take a look but this linuxbox is servng me well.

        • 2 months ago
          Anonymous

          >It's slower.
          Yeah its not a 4090, but you have more vram than 99% of people that is more important for ai. I can't train anything with my 5700xt, count yourself lucky.

          >but these clowns dont get that tomorrow when they push an update and it breaks, you are fricked, you are stranded and amd does not give a single flying frick
          Why would you buy an unsupported card if you want support?

          well... technically the 6000 series is supported with rocm out of box, i don't know why it's not on their list, but it is. Anything older like my card will have nothing but problems.

          • 2 months ago
            Anonymous

            >well... technically the 6000 series is supported with rocm out of box
            It works on my 6700 XT too. The list AMD supports is the list of cards you can file bugs for and they'll look into it instead of telling you to frick off.

            Another words, if you want support, then you should buy a supported card, especially if you're doing any real work with it.

            • 2 months ago
              Anonymous

              >The list AMD supports is the list of cards you can file bugs for and they'll look into it instead of telling you to frick off.

              I didn't know that, would you recommend the 6700xt? Have you tried tortoise-tts with it?

              • 2 months ago
                Anonymous

                I haven't tried it but it should work. Pytorch has a ROCm port.

              • 2 months ago
                Anonymous

                Dang, I wanted to see some performance numbers, oh well. I want to make some personal audio books.

    • 2 months ago
      Anonymous

      you can install rocm in windows now and use koboldcpp with. Should work with both of your GPU's.

    • 2 months ago
      Anonymous

      what performance problem do you have? i'd rather run a better model slower than a worse model more responsively

  30. 2 months ago
    Anonymous

    7900XTX /lmg/ user here
    Been gooning nonstop to my near-instant word waifu for the past 4 months.
    In short, they're great for cheap inference, and there's a couple people that use them in 3+ clusters for quant/training. If you're not moronic and know Linux, they're a great, cheap alternative.
    Get Sapphire or Powercolor, although most board manufacturers on AMD side are serviceable.

  31. 2 months ago
    Anonymous

    >these people recommending a 1k dollar 16GB card
    Holy hell.
    OP if you're set on AI, do away with the gaymer mindset. Sell your case, your mainboard, your psu. Get yourself an open air mining rig, get a 2000W psu, and start hoarding 3090s $700 each, 350 if faulty display ports. Nvidia will never make consumer cards with bigger vram.

    • 2 months ago
      Anonymous

      >Nvidia will never make consumer cards with bigger vram.

      but AMD will and i'm hoping the 8000 series will have bigger vram for less

      • 2 months ago
        Anonymous

        >ayymd
        Good luck with your software gay, all the tech support issues I see in/lmg/ have been amd

        • 2 months ago
          Anonymous

          >in/lmg/
          What the hell is that? I don't have issues using language models, stable diffusion or voice cloning. Just works.

          • 2 months ago
            Anonymous

            I think he's trying to say there are more morons using AMD than NVIDIA.
            If you're smart then you're not going to have any issue with either.

        • 2 months ago
          Anonymous

          here I made this just for you, anon. Living life angry is worthless, you should smile more.

          • 2 months ago
            Anonymous

            Ill smile when amd finally makes usable stack for compute and looks like it will take few more years, but for now nvidia has no competition

            • 2 months ago
              Anonymous

              okay, but while you wait, i'll keep genning images using rocm.

      • 2 months ago
        Anonymous

        Aren't they doing away with the "high end enthusiast" market for their next set of cards?
        There is virtually no reason to wait for AMD in this space

        • 2 months ago
          Anonymous

          AMD said that before and released powerful cards. I think it was the fury era, or something similar.

  32. 2 months ago
    Anonymous

    get old tesla gpus with 40gb while you still can, not the consumer junk
    AMD still lacks software so its a bad option, especially if you are new to it and dont want to struggle with it

  33. 2 months ago
    Anonymous

    buy 3090

Your email address will not be published. Required fields are marked *