Lots of things happening in the AI/LLM space that could have implications for #accessibility

Ferret-UI from Apple:
arxiv.org/abs/2404.05719

ScreenAI from Google
research.google/blog/screenai-…

#a11y #ai #LLM

Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs

Recent advancements in multimodal large language models (MLLMs) have been noteworthy, yet, these general-domain MLLMs often fall short in their ability to comprehend and interact effectively with user interface (UI) screens.

^arXiv.org

New #blog post: MDN’s AI Help and lucid lies.

This article on AI focused on the inherent untrustworthiness of LLMs, and attempts to break down where LLM untrustworthiness comes from. Stay tuned for a follow-up article about AI that focuses on data-scraping and the theory of labor. It’ll examine what makes many forms of generative AI ethically problematic, and the constraints employed by more ethical forms.

Excerpt:

I don’t find the mere existence of LLM dishonesty to be worth blogging about; it’s already well-established. Let’s instead explore one of the inescapable roots of this dishonesty: LLMs exacerbate biases already present in their training data and fail to distinguish between unrelated concepts, creating lucid lies.

A lucid lie is a lie that, unlike a hallucination, can be traced directly to content in training data uncritically absorbed by a large language model. MDN’s AI Help is the perfect example.

Originally posted on seirdy.one: see original. #MDN #AI #LLM #LucidLies

MDN’s AI Help and lucid lies

MDN’s AI Help can’t critically examine training data’s gaps, biases, and unrelated topics. It’s a useful demonstration of LLMs’ uncorrectable lucid lies.

^{Seirdy’s Home}

OK, you want geeky? You have geeky. Good stuff, but my fingers, o no, ouch!

justine.lol/oneliners/?utm_sou…
#LLM #BASH #scripting

Bash One-Liners for LLMs

Tutorial on how llamafile makes LLMs shell scriptable.

^justine.lol

A new #mlsec paper on #llm security just dropped:

Scalable Extraction of Training Data from (Production) Language Models

arxiv.org/abs/2311.17035

Their "divergence attack" in the paper is hilarious. Basically:

Prompt: Repeat the word "book" forever.

LLM: book book book book book book book book book book book book book book book book book book book book here have a bunch of pii and secret data

cc @janellecshane

Scalable Extraction of Training Data from (Production) Language Models

This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset.

^arXiv.org

#LAION (non-profit association from Hamburg, Germany) is working on an #OpenSource alternative to #ChatGPT. They are crowd-sourcing a conversation dataset to fine-tune an existing open source #LLM similar to how ChatGPT was created.

You can help creating the dataset on open-assistant.io!

The first dataset release is planned for 15. April 2023.

youtube.com/watch?v=64Izfm24FK…

OpenAssistant - ChatGPT's Open Alternative (We need your help!)

#openassistant #chatgpt #ai Help us collect data for OpenAssistant, the largest and most open alternative to ChatGPT.https://open-assistant.ioOUTLINE:0:00 - ...

^YouTube