ikt@aussie.zone to LocalLLaMA@sh.itjust.worksEnglish · 1 month agoNVIDIA's GeForce RTX 4090 With 96GB VRAM Reportedly Exists; The GPU May Enter Mass Production Soon, Targeting AI Workloadswccftech.comexternal-linkmessage-square2fedilinkarrow-up11arrow-down10file-text
arrow-up11arrow-down1external-linkNVIDIA's GeForce RTX 4090 With 96GB VRAM Reportedly Exists; The GPU May Enter Mass Production Soon, Targeting AI Workloadswccftech.comikt@aussie.zone to LocalLLaMA@sh.itjust.worksEnglish · 1 month agomessage-square2fedilinkfile-text
minus-squarehendrik@palaver.p3x.delinkfedilinkEnglisharrow-up0·edit-21 month agoHmmh, the 4090 is kind if the wrong choice for this, due to its memory bus width… For AI workloads and especially if you want to connect lots of memory, you kind of want the widest bus possible.
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up0·edit-221 days agoIt’s 384 bit? It’s not bad, 512 bit is super expensive and basically only exists on the 5090 die. Also, it seems LLMs are drifting towards being less memory-speed bound with the diffusion model experiments.
Hmmh, the 4090 is kind if the wrong choice for this, due to its memory bus width… For AI workloads and especially if you want to connect lots of memory, you kind of want the widest bus possible.
It’s 384 bit? It’s not bad, 512 bit is super expensive and basically only exists on the 5090 die.
Also, it seems LLMs are drifting towards being less memory-speed bound with the diffusion model experiments.