Adventures in AI

OK so I am pretty against AI for all the reasons you all know. But my PhD supervisors, including the head honcho of our lab, are using it and the university recently sent round guidance as to how you can use it in your studies.

I had a tedious task to do, cross checking citations against references in a long review paper. Then I had to check for typos and grammatical errors. My stepdaughter said, “ChatGPT could do those for you easily” and I thought, come on then, let’s fucking test it.

Task 1: Cross checking citations and references. ChatGPT standardised my referencing so everything was formatted correctly in APA style, which was helpful. It also threw away 2/3 of my references, even though they were cited in the text, which… wasn’t.

Task 2: Proofread. ChatGPT went further and asked me if I wanted it to produce a clean, proof-read version of my paper, ready to submit to a journal. I said, sure, yes please. ChatGPT said it would get back to me with my file in a few minutes.

Half an hour later I asked ChatGPT what was taking so long. It responded by telling me, “Thanks, Charlotte - you’re awesome! 💛” then asking me whether I wanted a completed Word doc or plain text. I chose the Word doc and ChatGPT said it would be ready in a minute.

Nothing.

About 15 minutes later I asked when it would be ready. ChatGPT thanked me for my patience and said it just needed to check whether I wanted a covering note for an editor to complement the article it was going to give me. I said yes please.

Nothing. I went to bed.

The next morning I said that I was tired of waiting and ChatGPT said (I paraphrase, but only slightly), “Here’s the thing, Charlotte. I can’t actually provide the version of the document I said I would produce for you ready for submission because it’s too complex a task for me. Would you like me to provide you with plain text of the article so you can do your own formatting?”

So I lost a bunch of references (obviously I had them saved elsewhere), got no further with my proofreading, and wasted about an hour of my time only to find out it had lied to me about its capabilities.

No need to say “I told you so.” At least I have been vindicated.

VScan is my little research project aiming to explore whether large language models (LLMs) with vision support could be used by blind people as a tool for performing visual cognitive tasks, and how useful could they be on travel and in day to day life. Our world is built on visual communication. Eyesight is a very rich and detailed sense, but most of the time, people are not scanning their field of view pixel by pixel, but they are purposefully looking for specific items or events in the environment, like a directional sign, the line number printed on a incoming bus, or their train appearing on the annoucement board of a train-station. Layouts of buildings and open spaces are communicated visually, contextual information is communicated visually, announcements and warnings are communicated visually.
What if we could bridge the visual communication of the world to our senses? VScan is trying to find out whether this is possible, feasible and whether and to what extend would it be useful. The entire project is open-source and tries to be as clear and transparent about the used technology as I'm able to communicate.

The recent months have been very important for VScan. In the new versions you can now easily use any OpenAI protocol compatible LLM server (including a self-hosted one), and run any common or uncommon model on it. The new system is very flexible, so you can configure everything to the tiniest detail and easily swap the used models or their providers. At the same time, the app comes with presets, which should (hopefully) make simple things simple and complex things possible.
Since this is a pretty large change, I highly recommend reading the initial setup part of the project's readme to understand the new system. If you're updating from v0.2 or earlier, you will need to do some configuration changes.
There are several reasons why this feature is so important. It makes VScan truly universal, not just in terms of the providers, but also the used technology. Now, VScan has the freedom to leverage the capabilities of the entire LLM market instead of just a single player, which brings some very interesting potential crucial for the next development.
It also gives you a complete control over your data, so you can decide the processing destination for each pixel you capture with your camera.

The new model system is powerful, but it's not the only added feature. There are many more, including a very convenient multipurpose text field on the scanning screen, a new action system, or a new prompt editor. Make sure to read the release notes for more details!

One more announcement. VScan has finally passed the review process for inclusion in F-Droid, and got also released in Google Play. You should now be able to find the app in your favourite app store.

Happy hacking!

github.com/RastislavKish/VScan

GitHub - RastislavKish/VScan: A visual perception layer for the blind

A visual perception layer for the blind. Contribute to RastislavKish/VScan development by creating an account on GitHub.

^GitHub