Skip to main content

What is Flash Attention huggingface.co/docs/text-gener… #llm #ai #ollama

Flash Attention

We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
#AI #llm #ollama
in reply to victor tsaran

Chi Kim
Chi Kim
• •
I wish their Python library supports Apple silicon like how llama.cpp implemented it. More and more models utilize it now, so you can't run them on Apple silicon with Python. :(
in reply to victor tsaran

Chi Kim
Chi Kim
• •
Hopefully soon! Flash attention has been out a while though. Even other vision language models have started using them. :( Makes me want to go get a rtx card for my pc. lol
in reply to Chi Kim

victor tsaran
victor tsaran
• •
@chikim Aren't they now talking about double-flash-attention, or something like that? I came across it yesterday, but didn't dig too deep into it just yet.
@Chi Kim
⇧