I've been on mastodon a few times over the years but this is the first time I've created an account with the intent of actually interacting. And I'd like to start by making everyone here hate me by sharing a clearly unpopular opinion and say: I... Like using AI. And I hope I can win some of you over as well.
Now don't get me wrong, I see the damage, I hate the people causing it, but I would ask, if we gave up on every new piece of technology that wasn't immediately as good as its predecessor, where would we be? We would never have invested in electric cars, or hell, vehicles at all. We wouldn't dare to building anything for fear our initial prototype couldn't compete with a refined legacy standard.
Many of the people on this platform are the people who actually can affect change. We're programmers, hackers, network engineers, the ones who actually get to do something. But instead of finding solutions, trying to make things better, so many are burying their head in the sand and hoping the world doesn't change around them. And I hate to say it but it won't work.
The world is changing, technology is changing, but we have the ability and knowledge to build the alternative. We're the ones who can challege big AI, and implement solutions built by people, outside of data centers, and in the control of people. As a person who has to implement these solutions for businesses, open source, local and edge applications are where I want to see this technology, not in a datacenter thats destroying a community.
I've done a lot over the years to push for open source technology, more than most can say they have. I didn't sit in a bubble and wait for the world to adopt the tools I want them to use, I climbed the ranks as from an IT professional to a sysadmin, a network engineer, and finally someone who actually has a say. And then I deployed those technologies for people. I gave end users linux, I chose proxmox over esxi, I deployed open source office suites instead of just paying for microsoft office because "that's what you do", even when it made my job harder.
I look at the landscape today in tech and I see an opportunity for us as open source devs and builders to shape things before they take place. To take the place of big AI in communities and businesses before it gains a foothold. And it hurts a bit to see businesses and organizations I've loved and followed for so long digging themselves a grave and praying it will all go away while the people like me have to keep helping businesses use chatgpt and gemini trash without good alternatives. Thank you for your attention, I'll be taking hate mail in the comments below.
reshared this
Das Vitamin C haltige Mark schmeckt schön fruchtig.
miki
in reply to Beb0p • • •Local AI wastes more power than AI that runs in a datacenter.
For LLMs to be as economical and power-efficient as possible, you need to keep your GPUs as highly utilized as you can, preferrably during all hours of the day. For big models, you also want to have as many of those GPUs as you can in one cluster, within limits of course.
THe thing about LLMs is that, unlike let's say Netflix, they require very little bandwidth and are relatively tolerant of latency. This means that there's little disadvantage (beyond the political implications) of running your model at the other end of the world. Unless you're a global organization with offices everywhere, if you buy your own GPUs, those GPUs will likely sit idle beyond 9-to-5. This won't be the case if you rent capacity from a global provider, as that provider can rent the same GPUs to somebody located in a different timezone when you're not using them.
Beb0p
in reply to miki • • •miki
in reply to Beb0p • • •You don't realize how much not a cost bandwidth is when it comes to AI. One Netflix movie (despite all the layers of CDNs) will probably offset your Chat GPT use for a year (napkin math).
I think you have a point re: community impact, but due to how much of a non-concern bandwidth is, we have a really great opportunity to build AI datacenters very close to green energy sources (or build green energy sources closer to AI datacenters). THis includes things like geothermal. If eveybody is running their own AI, they'll use whatever power they have.
This is one of a very few industries that a) uses insane amounts of energy, and b) the energy can be used at the other end of the world from where the product user is, with 0 loss to the user experience.
miki
in reply to miki • • •