Seems pretty underwhelming. They’re comparing a 109B to a 27B and it’s kind of close. I know it’s only 17B active but that’s irrelevant for local users who are more likely going to be filtered by memory rather than speed.

200 tokens per second isn’t achievable with a 1.5B even on low-midrange GPUs. Unless they’re attaching an external GPU it’s not happening on a raspberry pi.

This article is disjointed and smells like AI.

Probably just a reporting bug. Comments stayed consistent.

exFAT is still the best format for multiplatform compatibility so it’s good to see that it’s still getting maintained.

The issue isn’t really “too busy with jobs”, it’s not having enough monetary resources to take care of a large non-working population. The fear with AI is that it will take the comfy, high paying office jobs and demote the non c-suites down to manual labor. Unless corporate taxes are increased dramatically (unlikely) many of the young working class will be stretched thin trying to pay for their parent’s care.

What’s the deal with Alpine not using GNU? Is it a technical or ideological thing? Or is it another “because we can” type distro?

I don’t think the term open-source can be applied to model weights. Even if you have the exact data, config, trainer and cluster it’s basically impossible to reproduce an exact model. Calling a model “open” sort of works but then there’s the distinction between open for research and open for commercial use. I think it’s kind of similar to the “free” software distinction. Maybe there’s some Latin word we could use.

Your best bet would probably be to get a used office PC to put the card in. You’ll likely have to replace the power supply and maybe swap the storage but with how much proper external enclosures go for the price might not be too different. Some frameworks don’t support direct GPU loading so make sure that you have more ram than vram.

An arm soc won’t work in most cases due to a lack of bandwidth and software support. The only board I know of that can do it is the rpi5 and that’s still mostly a poc.

In general I wouldn’t recomend a titan x unless you already have one because it’s been deprecated in cuda, so getting modern libraries to work will be a pain.

I really like the simplicity and formatting of stock pacman. It’s not super colorful but it’s fast and gives you all of the info you need. yay (or paru if you’re a hipster) is the icing on top.

It does a little bit worse than v0.1 on all benchmarks which isn’t ideal. That doesn’t really say much about the finetuning potential though.

Don’t buy a Chromebook for linux. While driver support usually isn’t an issue, the alternative keyboard layout is terrible for most applications. To even get access to all of the normal keys that many applications expect you need to configure multi-key shortcuts which varies in complexity based on your DE. In most cases it will also void your warranty because of the custom firmware requirement.

Test comment

This is big if true, but we’ll have to see how well it holds up at larger scales.

The size of the paper is a bit worrying but the authors are all very reputable. Several were also contributors on the retnet and kosmos2/2.5 papers.

From my experience the model is pretty bad compared to both 7B llama2 and mistral. It’s also larger than both. It might be caused by bad instruction tuning, but overall my expectations are pretty low.

Koboldcpp should allow you to run much larger models with a little bit of ram offloading. There’s a fork that supports rocm for AMD cards: https://github.com/YellowRoseCx/koboldcpp-rocm

Make sure to use quantized models for the best performace, q4k_M being the standard.

Tun0 is the interface that most vpns are using so I assume proton is the same.

It’s not open source it’s weight available(for now). As of now there’s nothing you can do with it publicly because it lacks a license and is known to be stolen.

Flatpak is good for diversity. Users don’t need to worry about whether the obscure distro they want to use has the software they want in its repos. If a distro supports flatpak it will work with most popular software out of the box.

French laws don’t recognize software patents so videolan doesn’t either. This is likely a reference to vlc supporting h265 playback without verifying a license. These days most opensource software pretends that the h265 patents and licensing fees don’t exist for convenience. I believe libavcodec is distributed with support enabled by default.

Nearly every device with hardware accelerated h265 support has already had the license paid for, so there’s not much point in enforcing it. Only large companies like Microsoft and Red Hat bother.