That’s cool! I’m really interested to know how many tokens per second you can get with a really good U.2. My gut is that it won’t actually be better than the 24VRAM+96RAM cache setup this user already tested with though.

How much do you need? Show your maths. I looked it up online for my post, and the website said 1747GB, which is completely in-line with other models.

https://apxml.com/posts/gpu-requirements-deepseek-r1

Thanks!

Can you link that post?

Running R1 locally isn’t realistic. But you can rent a server and run it privately on someone else’s computer. It costs about 10 per hour to run. You can run it on CPU for a little less. You need about 2TB of RAM.

If you want to run it at home, even quantized in 4 bit, you need 20 4090s. And since you can only have 4 per computer for normal desktop mainboards, that’s 5 whole extra computers too, and you need to figure out networking between them. A more realistic setup is probably running it on CPU, with some layers offloaded to 4 GPUs. In that case you’ll need 4 4090s and 512GB of system RAM. Absolutely not cheap or what most people have, but technically still within the top top top end of what you might have on your home computer. And remember this is still the dumb 4 bit configuration.

Edit: I double-checked and 512GB of RAM is unrealistic. In fact anything higher than 192 is unrealistic. (High-end) AM5 mainboards support up to 256GB, but 64GB RAM sticks are much more expensive than 48GB ones. Most people will probably opt for 48GB or lower sticks. You need a Threadripper to be able to use 512GB. Very unlikely for your home computer, but maybe it makes sense with something else you do professionally. In which case you might also have 8 RAM slots. And such a person might then think it’s reasonable to spend 3000 Euro on RAM. If you spent 15K Euro on your home computer, you might be able to run a reduced version of R1 very slowly.

It’s the 671B model that’s competitive with o1. So you need 16 80GB cards. The comments seem very happy with the smaller versions, and I’m going to try one now, but it doesn’t seem like anything you can run on a home computer with 4 4090s is going to be in the ballpark comparable to ChatGPT.

The user’s library of apps are Windows apps. And Windows does support Linux programs. There are versions of Windows that don’t technically have it enabled by default, but it’s easy to install support. It has a built in command “wsl --install”, and a button in the store and start-menu. And for most users who get a pre-configured image from IT or their laptop manufacturer it’s pre-installed.

I really don’t get this community’s insistence on getting people to use Linux no matter how much destruction they bring. Steam games on Linux are not what anyone has in mind when they say Linux doesn’t have games. Because Linux isn’t binary compatibility, it’s libre software.

In my circles, if someone says “Linux such and such”, we assume they might be referring to their FreeBSD computer as well. Here it seems Linux is more likely to refer to Android. Emulating a sketchy Windows game doesn’t make Linux the better platform for games. The Windows games are always going to be best on Windows, and now your Linux computer has malware on it.

thought-terminating cliché

There’s no argument, it’s the definition of the word. Why do you assume there should be argument around the normal usage of a word?

Google Play get fined and the reward goes to Google Android.

I used to use Gnome with a tiled window manager. It was a good combo. Don’t see why they have to be exclusive. No hate from my side, KDE and Gnome are both incredible. I can spare some hate for the Gnome-haters though.

The Akkoma instance hosted on kernel.org
https://social.kernel.org/notice/AWSXomDbvdxKgOxVAm

No part of open source puts value in collaboration and democratising the means of the production. Free software is definitely not about reducing inherent contradictions and exploitation that arise from your livelihood being dependant on someone else’s private property.

Though sometimes you get confused randos like this saying stuff they don’t understand, probably where the confusion stems from.

Communism and Linux are completely unrelated.

Sounds like the instance admins should contribute. Hiring a third person to sneak ‘moderation tools and data privacy law related issues’ related development ahead in the queue would probably go a long way in stopping to ‘refuse to work’ on it. Or they could straight up actually help build the software.

White man telling you VPNs are not illegal. https://yewtu.be/watch?v=bzti2CELXrg

You don’t need to support Putin to be a Marxist, him and Russia are deserving of a lot of criticism, but it’s maybe a little suspicious if someone sees everything happening geopolitically right now and consistently chooses to focus their anger towards Russia.

To be honest when I read Supavillain’s quotes my first reaction was also 'So they don’t like Russia, who cares?", Ukrainian national identify exists, if Russia annexed western Ukraine they wouldn’t be able to erase it. Non-principled populist politicians do always suck. And Russia would’ve loved (I assume this is about) Donetsk PR to have been their puppet.

But then those last two quotes are pretty bad. Killing Putin or even having Putin do a 180 wouldn’t change anything. He’d just be replaced. And the poor countries are no less civilised.

It’s because China needs help using Emacs. Lemmy needs to rebrand to a Vim learning resource.

So what exactly is this? Open-source ChatGPT-alternatives have existed before and alongside ChatGPT the entire time, in the form of downloading oogabooga or a different interface and downloading an open source model from Huggingface. They aren’t competitive because users don’t have terabytes of VRAM or AI accelerators.

That tweet is so weak, how are hundreds of people here upvoting and commenting on this?