As per a recent leak, the RTX 4090 with higher VRAM than 24 GB exists. This includes a new 96 GB variant, which a factory is working on.

Maybe AMD’s loss is Nvidias gain ?

  • hendrik@palaver.p3x.deEnglish
    0·
    1 month ago

    Hmmh, the 4090 is kind if the wrong choice for this, due to its memory bus width… For AI workloads and especially if you want to connect lots of memory, you kind of want the widest bus possible.

    • brucethemoose@lemmy.worldEnglish
      0·
      21 days ago

      It’s 384 bit? It’s not bad, 512 bit is super expensive and basically only exists on the 5090 die.

      Also, it seems LLMs are drifting towards being less memory-speed bound with the diffusion model experiments.