It’s no secret that many of us in the blind community have embraced the rapid advances in Artificial Intelligence over the past two years. We've witnessed firsthand how these technologies can be a powerful force for good, especially within our community. AI-generated image descriptions have revolutionized how we navigate the online world, offering a perspective previously unimaginable. This impact is now undeniable, transforming how we interact with the world.”

I’ve declared the kingdom of the blind a republic—perhaps prematurely, but only by a small margin. With AI empowering us to perceive the digital world in new ways, we are no longer ruled by limitations, but actively shaping our future. Anthropic’s recent launch of ‘computer use’ marks the first steps into a new phase of AI evolution—one where AI agents begin to act independently on our behalf, initiating a shift in how we interact with technology.

As AI continues to evolve, so too will the Assistive Technology that many of us depend on. I envision a future where this intelligence becomes a true companion, guiding us seamlessly through both digital landscapes and real-world challenges. We may be just two years away from seeing JAWS, NVDA, or SuperNova transform into true Assistive Intelligence 1.0—or perhaps it will take a little longer. If AI has taught us anything, it’s that progress comes both more slowly than we expect and faster than we can possibly imagine.

What follows is my first attempt at describing how a screen reader of today could take the first steps towards becoming an Assistive Intelligence. If anyone wants to build it, I’d love to help if I can. Whatever you think, let me know what you think:

“Proposed AI-Powered Self-Scripting Feature for JAWS Screen Reader

Objective
The suggested feature seeks to integrate advanced AI-driven "computer use" capabilities, like those developed by Claude (Anthropic), into the JAWS screen reader. This functionality would enable JAWS to autonomously create and refine custom scripts in response to real-time user interactions and application environments. The aim is to enhance accessibility and productivity for visually impaired users, especially when navigating non-standard or otherwise inaccessible software interfaces.

Feature Description
The self-scripting capability would empower JAWS to analyse user interactions with applications, identify recurring actions or inaccessible elements, and generate scripts that optimize these processes. By enabling JAWS to perform this autonomously, users gain seamless and personalized access to applications without manual intervention, allowing for an enhanced, efficient experience.

The self-scripting feature will be powered by the following core functions:

1. Real-Time Autonomous Scripting: JAWS would use AI to observe user interactions with applications, especially non-accessible ones, and automatically generate scripts that improve navigation, label untagged elements, and streamline frequent tasks. For example, if a user frequently navigates to a particular form field, JAWS could create a shortcut to this area.

2. Adaptive Behaviour Learning: This feature would allow JAWS to recognize patterns in a user’s interactions, such as repeated actions or commonly accessed elements. JAWS would adapt its behaviour by creating custom macros, enabling faster navigation and interaction with complex workflows.

3. Dynamic Accessibility Adjustment: Leveraging Claude’s approach to visual recognition, JAWS could interpret visual elements (like buttons or icons) and provide instant labelling or feedback. This would be valuable in software with minimal accessibility features, as it enables JAWS to make live adjustments and effectively “teach itself” how to navigate new environments.

4. Community Script Sharing: Self-generated scripts, once verified, could be anonymized, and made available to other users via a shared repository. This would foster a collaborative environment, empowering users to contribute to a broader database of accessibility scripts for applications across various industries.

Value Proposition
This feature will address key challenges for visually impaired users, including the complexity of navigating inaccessible interfaces and the time-consuming nature of repetitive tasks. The ability for JAWS to generate its own scripts autonomously would mean:
1. Increased Accessibility: Improved interaction with non-accessible software interfaces.
2. Higher Productivity: Reduced need for external support or manual scripting, allowing users to accomplish tasks more independently.
3. Enhanced User Experience: Scripting and macro creation based on personal usage patterns -- leads to a more intuitive and personalized experience.

Technical Considerations
1. Performance: Processing real-time visual and user interaction data requires substantial computing power. A cloud-based model may be optimal, offloading some processing requirements and ensuring smooth, responsive performance.
2. Safety: Automated scripting must be closely monitored to prevent unintended interactions or conflicts within applications. Integration of safeguard protocols and user settings to enable/disable autonomous scripting will be essential.
3. Privacy: To ensure user data is protected, anonymization protocols and data privacy standards will be implemented. Data collected from user interactions would be handled in compliance with rigorous privacy standards, safeguarding user preferences and behaviour.

Conclusion
Integrating AI-powered self-scripting capabilities into JAWS would represent a significant leap in screen reader technology. By allowing JAWS to, when requested, autonomously learn, adapt, and script in response to user needs, this feature could provide visually impaired users with unprecedented control and flexibility in navigating digital environments, fostering both independence and productivity. The anticipated benefits underscore the feature’s potential to redefine accessible technology, turning screen reader into Assistive Intelligence.“

About the Author:

Lottie is a passionate advocate for the transformative potential of AI, especially within the blind and visually impaired community. She blends technical insights with a keen awareness of lived experiences, envisioning a future where AI doesn’t just assist but truly empowers. Her thoughtful reflections explore the shift from a "kingdom of the blind" to a republic, where emerging technologies like AI create new opportunities for autonomy and inclusion.

With a balance of optimism and critical realism, Lottie acknowledges the game-changing impact of AI tools like image descriptions while recognizing that more progress is needed. Her vision extends to the idea of "Assistive Intelligence," where screen readers like JAWS evolve into proactive companions, adapting to users' needs in real-time.

Known for turning complex ideas into actionable blueprints, Lottie is not just an observer of technological trends but a catalyst for innovation. Her proposals reflect a desire to elevate independence and productivity for blind users, pushing the boundaries of what's possible in assistive technology. Her insights continue to inspire conversations and shape the future of accessible tech.

I am the Blind AI, relying on AI every day to enrich my life. While my posts may occasionally benefit from AI assistance, the thoughts, perspectives, and final edits are entirely my own. AI is my tool, much like a calculator or spell-check, refining my expression but never replacing my voice.

#Accessibility #AI #AIsoftheBlind #Blind #ComputerVision #Disability #Innovation #JAWS #NVDA #ScreenReader #SuperNov

Shopify, accessibility lawsuits & 'Compliance'

buff.ly/3YfG7bn

#eCommerce #Shopify #DigitalAccessibility #Accessibility #A11y #ACR #VPAT

I discovered a great new language learning platform yesterday. One of its key workflows presents content in a dialog, and it works great with NVDA. When the dialog opens in Safari on iOS, though, VoiceOver decides that the entire screen is blank.

I'm so tired of #accessibility workarounds being needed for everything in life. Trying to apply my mental energy to something useful and interesting, only to end up spending it on the same old shit is exhausting.

#accessibility question:

Does anyone know if a HTML support for Orca screen-reader exist?
(like @SteveFaulkner's tests on github.com/stevefaulkner/scree…)

#Orca #screenReader #a11y #Linux

GitHub - stevefaulkner/screen-reader-HTML-support

Contribute to stevefaulkner/screen-reader-HTML-support development by creating an account on GitHub.

^GitHub

Rolling out to #Pachli Current now:

- Long-press links, hashtags, and mentions to open the "share sheet"
- Better UI when writing media captions
- #Accessibility fixes:
. "Open boost author" works as intended
. Include a "Copy" button for links/hashtags/mentions in dialogs
. Show links/hashtags/mentions options in notifications
. Use checkboxes when managing list membership

Most of these were from user feedback. Thanks to everyone who took the time to provide it.

For those who want a peek behind the scenes – our #accessibility team’s yearly retrospective notes are all public: github.com/wagtail/wagtail/dis…

Highlights of what’s coming up next:

- Accessibility x #django workshop proposals @oscafest and @djangoconafrica 2025
- More testing with our users with disabilities
- More automated tests
- Continue the data-driven accessibility decisions

Accessibility team: meeting updates · wagtail wagtail · Discussion #11394

👋 Subscribe to this thread for regular updates on what our accessibility team is up to, unsubscribe if this isn’t for you (in "Notifications" to the side 👉). 2024-10-18 – retrospective 2024-10-04 –...

^GitHub

Wonderful article from @bas on getting started with #accessibility. basbroek.nl/making-accessibili…

I love how he emphasizes that while accessibility is a vast space, we can start small and learn from the assistive technologies already built into our devices!

A big shout out to everyone at #A11yCamp yesterday. Sean spent a very informative day there leaning and networking on all the latest in #A11y / #Accessibility!

Photos are Sean with Riley Busch from the department of education, and a view of the Clarendon Auditorium filling up before the event started

As noted on the banner in the first photo, A11y Camp is Australia's premier conference on digital accessibility and inclusion.

Right, I'm going to do something I dislike, but I literally don't know how else to get support to follow up with me.
I have indeed submitted two support requests in the intervening time, i'm not just being lazy I promise.
On 17 Aug I made a video about #FloatPlane accessibility and how not good it is.
I wasn't a troll, I was constructive and thoughtful, and yet I cannot get a hold of anyone at either Floatplane Media or LTT.
If *anyone* happens to know someone at one of these companies and could get my video to them, I'd be extremely grateful.
The other thing I really dislike doing is asking if people could boost for reach, but I'll make an exception.
Fedi is big now, and far more reaching than modern-day xitter, so all help gratefully received.
I thank you in advance.

Accessibility Issues With Floatplane (from a VoiceOver Perspective) youtu.be/2sZTsrek08M
#LTT #LinusTechTips #Accessibility #VoiceOver #iOS

Accessibility Issues With Floatplane (from a VoiceOver Perspective)

I've never made a video I wanted to age as badly as this one, but needs must.There are some glaringly obvious issues with LTT's Floatplane app for iOS if you...

^YouTube

@NVAccess posted:

In-Process 23rd October 2024

nvaccess.org/post/in-process-2…

#NVDASR #accessibility

In-Process 23rd October 2024

Over the past fortnight we had a fantastic response to our braille survey, thank you! We already have some learnings to share with you! Plus, we’re nearly ready for NVDA 2024.4, are you? We’ve also…

^{NV Access}

Alt text isn't just for the vision impaired. Sometimes some of us are viewing an image and don't understand the context or some of the imagery. Alt text helps there too.

#accessibility

Für @nextcloud@mastodon.xyz habe ich große Sympatie.

Sie wird allerdings mehr und mehr auf die Probe gestellt.

* Die Desktop- und Android-Clients, die nach dem Update wieder mal alles neu synchronisieren möchten
* Das Deck welches dringend #UI und Funktionalitätsverbesserungen verträgt
* Die Nautilus-Desktop-Erweiterung, die für Blinde und Sehgeschädigte nahezu unbenutzbar ist und für Sehende nicht viel besser

... um nur ein paar Beispiele zu nennen.

Wären Arbeiten an den Basisfunktionalitäten nicht wichtiger als immer neue Hubs mit irgendwelchem AI-Zeugs?

#Nextcloud #a11y #Accessibility #basics

Related question: I'm familiar with Atkinson Hyperlegible. the Braille Institute Free Font (link at the bottom.

If accessibility-oriented fonts make a difference for you as a reader (for dyslexia, low vision, visual processing, or any other reason), I would love to know what accessible fonts you personally like best and why.

*This is question for people who rely on accessible fonts only. No replies from others, please. Thank you!

brailleinstitute.org/freefont/

#Accessibility #A11y #Fonts

Atkinson Hyperlegible Font - Braille Institute

Read easier with Atkinson Hyperlegible Font, crafted for low-vision readers. Download for free and enjoy clear letters and numbers on your computer!

^{Braille Institute}

I have a question for people who work in accessibility and design:

Can anyone recommend evidence-based sources on how well Comic Sans does or doesn't work for people with dyslexia?

- I don't need anecdotes, opinions, or big feelings about fonts.
- I'm just looking for evidence-based studies.

I've seen strong, contradictory opinions & a lot of received wisdom about this. That's why I'm looking for actual evidence either way. Thank you!

#Accessibility #AccessibleDesign #Design #A11y #ComicSans

Die #ubuntu 24 Neu #Installation scheint vor Veröffentlichung nicht einmal ohne Verwendung der Maus getestet worden zu sein...

#a11y #Accessibility #Linux

Do you rely on alt text on social media?

#AltText #accessibility #SocialMedia #poll

• Yes, because of a disability (6%, 13 votes)
• Yes, it helps add context (60%, 116 votes)
• Yes (other reasons) (16%, 31 votes)
• No (33%, 64 votes)

@vick21 @Tamasg

I hope so too. #Accessibility is important -- especially for an app based on music

I've talked to a couple of screen reader users, who have bravely volunteered to test out early versions of this code

They've told me that it works well (mostly) and I'm still running through their feedback to make things better

So I won't say Bandwagon is 100% WCAG compliant, but please try it out and let me know if you have any trouble. I jump on accessibility issues as quick as possible

Summarizing the state of the last few months of #accessibility work, slowly but surely getting a full stack with no vision support across all layers from CLI shell to window management to cooperative clients to uncooperative clients.

arcan-fe.com/2024/10/18/access…

Accessible Arcan : Out of Sight

Our reference desktop environment, Durden, rarely gets covered here these days. This is mostly due to the major features are since long in place and that part of the project is biding its time with…

^Arcan

one of my real pain points with the iPhone is composing emails using the built-in mail app with #voiceover. It’s got slightly better with iOS 18, but the Misspelt Words option in the rotor still does not work. This might not be an issue for some, but for me I cannot send emails on the phone with confidence that it is written accurately and looks professional. I know there's the option of writing it in the Notes app, reviewing it and then pasting it into Mail, but come on! That's such a clunky workaround.

Is Mail any better on the iPad at all? I would quite like something that is an in between device to a laptop and a phone that I could use to quickly write emails, make additions to documents, write quick notes etc, and in that sense the iPad could be a nice fit, but if iPadOS has the same issues as the iOS versions then meh, it might not add much for me. Any thoughts?

#Blind #BlindMastodon #accessibility #a11y #Apple #iPad #iPhone #Productivity

@modulux @swearyanthony Whilst #touchscreens can increase #accessibility and #longevity (no button / force needed) they also can hinder intuitive operation.

• There's a resson we have buttons and knobs and levers in cars, ships, airplanes and even on bikes and scooters: You don't want to click through menues to be able to turn off the radio, activate warning lights or pull over...

You want immediate and blind control...

FINALLY! Open-source LIVE CAPTIONS for your Linux desktop that run locally! No internet connection, no spyware!

github.com/abb128/LiveCaptions

FUTO is a company that grants young developers means to build awesome tools like this one. Tools that help digital sovereignty.

futo.org/projects/

All I need now is a model that will translate all the Russians in #CounterStrike for me :D

#Accessibility #HearingAid #Deaf #FUTO #FreeSoftware #FairSoftware #MachineLearning

👁️ @w3c AGWG (Accessibility Guidelines Working Group) Teleconference
15 Oct 2024 - draft minutes

"RESOLUTION: We will explore having a set of required provisions that is somewhat comparable to WCAG 2.2 A & AA and a set of provisions and assertions that can be used to build upon that through points, percentages, or predefined sets of provisions (modules)."

#WCAG3 #WCAG #accessibility #WebStandards

w3.org/2024/10/15-ag-minutes.h…

Not so short note on aria-label usage – Big Table Edition

“aria-label is one of a number of secondary methods to label native HTML UI elements. It works particularly well on interactive elements, it also works well on most block level elements old skoolterm that have explicitly or implicitly defined structural roles. It works less well or not at all on text level semantics.”

#aria #HTML #accessibility #WebDev

html5accessibility.com/stuff/2…

Ever find that sitting in front of your unnecessarily complex music rig is hard sometimes? Creative block hits and there's nothing you can do about it?
I certainly find that lately, more often than not.
One device has come into my life and changed a lot of that however. Ableton Move.

In this world-first video, I take you through making a beat without sight, just using the undocumented screen-reader function within the web-based Move Manager.

It's incredibly freeing to be able to just load a fresh set, be presented with four random sounds and perhaps one of them will inspire you so you just begin doing a thing that you had absolutely no plan to do before you started.
#InspiredBySound - Let's Move! (Ableton Move Accessibility Overview) youtu.be/p8IbinbOhY4
#Accessibility #Ableton #ScreenReader #Blind #Music #Composition

#InspiredBySound - Let's Move! (Ableton Move Accessibility Overview)

Check out the official Ableton Move website: https://www.ableton.com/en/move/Ableton Move is the first piece of hardware I've personally been excited by in y...

^YouTube