While the #GenAI news cycle keeps announcing new models, cost and evaluation continue to be crucial for both developers and businesses.
This post showcases #OSS tools that help evaluate models while keeping costs low. We include Prometheus by KAIST AI; @MozillaAI's very own lm-buddy; and llamafile.
Davide Eynard @mala shows how these components can work together to evaluate LLMs on cheap(er) hardware.
blog.mozilla.ai/local-llm-as-j…
Local LLM-as-judge evaluation with lm-buddy, Prometheus and llamafile
In the bustling AI news cycle, where new models are unveiled at every turn, cost and evaluation don’t come up as frequently but are crucial to both developers and businesses in their use of AI systems.Davide Eynard (Mozilla.ai Blog)