Skip to main content


Summarizing the state of the last few months of #accessibility work, slowly but surely getting a full stack with no vision support across all layers from CLI shell to window management to cooperative clients to uncooperative clients.

arcan-fe.com/2024/10/18/access…

in reply to Bjorn Stahl

Thanks for working on this. I remember now that you sent me a private message on Lobsters about AccessKit, and I'm sorry I didn't follow up on that. What are you hoping that AccessKit will let you do?
in reply to Matt Campbell

Also, I listened to some of the first audio demo. The gaps between words are stranger, and almost certainly not what most blind people who are accustomed to fast speech would want. Also, are the words spoken with higher pitch capitalized, or something else?
in reply to Matt Campbell

@matt I didn't find any documents for what would be 'beginner' friendly settings for initial pitch/wpm/punctuation controls, so I just took what I mostly been training using and slowed it down.

The default_eng profile used 180wpm, full punctuation, mid dynamic range, only left channel and higher pitch on capitals.

in reply to Bjorn Stahl

For the default profile, I suggest removing the gap between words, disabling the higher pitch on capitalized words, and reducing the punctuation level (in particular, no end-of-sentence punctuation).
in reply to Matt Campbell

@matt since the TTS engine is already active as that stage I'll probably define a few presets and let the picker just read a reference text with the corresponding settings.

I'm taking a much needed break from hearing 'wnd tit chromium' about 50 times an hour to poke at other parts (audio transcription and low-vision). The best write-up I've seen is w3.org/TR/low-vision-needs -- any other recommendations?

in reply to Bjorn Stahl

I definitely don't recommend "wnd tit" as a prefix for window titles. If it's literally pronounced like that, including the letter w, then that puts six syllables before the window title. Putting something long and static before the variable text isn't a good idea. You could make it say "Chromium window", or maybe play a sound effect concurrently with reading the window title.
in reply to Matt Campbell

@matt from my reading of accessKit code it would be in the sense of a platform where there's a dynamic check if arcan_shmif_primary(SEGID_ACCESSIBILITY)) gives you a context, and use that to either send per-item updates (MESSAGE events) in response to input events or the full tree wrapped in a file descriptor as BCHUNKSTATE events. The later would leave navigation up to WM, the former closer to what others would do.
in reply to Bjorn Stahl

What AccessKit would give you, assuming you don't want to implement the existing AT-SPI protocol based on D-Bus (I don't recommend that), is the schema for an accessibility tree, including incremental tree updates. AccessKit doesn't directly support sending direct TTS messages (assuming that's what you meant by MESSAGE events), though your platform adapter could do that; I guess that would amount to implementing a screen reader running inside the application.
in reply to Matt Campbell

I remember reading that you intended to implement support for the Wayland protocol. You may be interested in my prototype Wayland protocol extension for accessibility, code-named Newton, and the reference toolkit-side implementation as an AccessKit platform adapter. That adapter is here: github.com/AccessKit/accesskit… More on the project here: blogs.gnome.org/a11y/2024/06/1…
in reply to Bjorn Stahl

what is this? I've been using linux for about 5 years already, yet never encountered this. Is this emacs related? it feels like it. Are you making a custom interface on top of the linux kernel? I have no idea what I just listened to. Why are the beeps repeating continuously? do you use your arrow keys to literally move the mouse around, isn't that slow? you have your own window management protocol? can I open gtk apps in it and get similar feedback to what you have there? I'm completely stumped as to what this even is, but imo, it's very interesting. For how long has this whole stack been a thing? I feel like it's something built on top of emacs, as I said before, is it? why did you have to press meta multiple times, and what are the screenreader modifiers, aka, can there be more than one?
in reply to the esoteric programmer

@esoteric_programmer How long it's been worked on? since early 2000s. How long it's been in public view? closer to ten years. It's not marketed as much as it is leaving behind a trail of crumbs.

See the index of articles: fossil.arcan-fe.com/ for how deep this hole goes.

Is it a custom interface on top if the linux kernel? Well anything above kernel level is fair game, though it also works on BSD kernels and, at least last I tested, OSX. It is about as 'different' architecturally as Android is - other IPC system, network protocol, Low level APIs, Mod-level APIs, several WMs..

The beeps are not continuous, they were mapped to the mouse cursor but binned to a timer instead of event driven as well, that triggered my tinnitus something fierce with a 1KHz mouse.

The meta modifier and a11y bindings being forced to be pressed repeatedly was to a. avoid accidental presses, b. cover for running nested (VM, inside another windowing system, ...) where a long press could get cancelled from keyrepeats not being correctly disabled or from hardware keys with contact bounce problems.

GTK applications would run through the arcan-wayland protocol bridge, or the Xarcan one (used in the clip with chromium) but those are so far just pixel soup so feedback comes from OCR and other sight to sound mapping.

in reply to Bjorn Stahl

I'm not sure if you can use rust, but if so, for now you can just make an at-spi viewer or however you called it there, the same way you'd do one for access kit, if nothing else, because access kit isn't implemented widely enough yet, and access kit translates to at-spi anyway. Here's the atspi crate we use as part of odilia, the same one access kit depends on. I dk how old the crates.io version is, so I'll give you the thing straight from the source

github.com/odilia-app/atspi

in reply to the esoteric programmer

@esoteric_programmer that's why I was chatting with @matt as AccessKit is architecturally more sound (Rust interop issues aside) - AT-SPI is a no-go simply because D-Bus is blacklisted on absolutely everything here.
in reply to Bjorn Stahl

you know access kit is using dbus as well, no? transpiling to at-spi, or using the newton protocol, it does require some dbus stuff. Also, how could you understand anything from that chromium part via ocr? I got lost from that point, and the fact that the mouse beeps were hurting my ears didn't help either :p
in reply to the esoteric programmer

@esoteric_programmer @matt not necessarily, that comes from the AT-SPI compatibility and Unix platform. I'd substitute in another platform for arcan-shmif and avoid that.

This was my first attempt at an aural only demo, and still no idea how to convey the haptic parts -- like moving the mouse around, combined with the wrist mounted piezo I use to feel when the cursor crosses horizontal/vertical edges.

I'm sighted (though this work has been blindfolded), so the audio profile used is what I learned to use with some notes from watching @zersiax 's twitch streams, yet as you hear, different enough from what others might be used to. That's also why I opted to not show any of the HRTF/positional audio or seeing-with-sound bits.

in reply to Bjorn Stahl

ahh, interesting, so you want to make your own platform for which accesskit builds an adaptor? I wonder, would that help the current situation with chromium and similar? whoe, you have that much support for esoteric devices like those? linux doesn't have them even now, not as advanced as your thing, awesome! Also, I know the name sounded familiar from somewhere, now I found it, this popped up in my articles list when I was researching something about wayland. Reading some of your building clients with arcan articles, especially the part where you apparently have to clone arcan and build your apps into its source tree somewhere to make them found by the thing, I came to the conclusion that you're using wayland in some very, very weird ways, dk if that's awesome or not. Anyway, this is the article I read back then, but as I had no context, I dismissed it because it sounded weird

arcan-fe.com/2017/12/24/crash-…