[Athen] Handwriting to OCR text

[Top Tech Tidbits via athen-list]

Please forgive the intrusion from someone who is NOT an expert in this matter. I just wanted to mention that we have had great success (as sighted persons at an ad agency) with using ChatGPT to recognize and translate handwriting by providing it with an image of said handwriting. Even entire sheets of it written in cursive. It’s pretty impressive. Better than what traditional OCR would provide of the same handwriting for sure. Again, no idea if this is relevant in any way, and if it is not, please forgive me, but I wanted to mention it just in case someone finds it useful. 😊

 

Aaron Di Blasi,  <https://www.pmi.org/> PMP

 

 <https://www.linkedin.com/in/aarondiblasi/> 

 

“The greatest barrier to accessibility is indifference.” 💡

 

PR Director (2024-Present)

AT-Newswire

Access Technology's Digital Newswire

 <https://at-newswire.com> https://at-newswire.com 🌐

 

Publisher (2024-Present)

AI-Weekly

The Week's News in Artificial Intelligence

 <https://ai-weekly.ai/> https://ai-weekly.ai 🌐

 

Publisher (2022-Present)

Access Information News 

The Week's News in Access Information

 <https://accessinformationnews.com> https://accessinformationnews.com 🌐

 

Publisher (2020-Present)

Top Tech Tidbits

The Week's News in Access Technology

 <https://toptechtidbits.com/> https://toptechtidbits.com 🌐

 

Sr. Project Management Professional (2006-Present)

Mind Vault Solutions, Ltd.

 <https://mvsltd.com/> https://mvsltd.com 🌐

 

Certified:

Digital Marketing Associate,  <https://mvsltd.com/news/aaron-di-blasi-pmp-mind-vault-solutions-ltd-awarded-digital-marketing-associate-certification-by-meta/> Meta Certified (2022 - Present)

Social Marketing Professional,  <https://mvsltd.com/news/aaron-di-blasi-pmp-mind-vault-solutions-ltd-awarded-social-media-marketing-certification-by-hootsuite-world-leader-in-social-media-marketing-solutions/> Hootsuite Certified (2020 - Present)

Email Marketing Professional,  <https://mvsltd.com/news/aaron-di-blasi-pmp-mind-vault-solutions-ltd-named-a-constant-contact-certified-solution-provider/> Constant Contact Certified (2019 - Present)

 

Specializing in:

Digital Strategy and Content Marketing

Social Media Advertising

Online Fundraising

 <https://www.ada.gov/> ADA,  <https://www.w3.org/WAI/standards-guidelines/wcag/> WCAG and  <https://www.justice.gov/crt/section-508-home-page-1> Section 508 Compliance

 

Website:  <https://mvsltd.com> https://mvsltd.com 🌐

Email:  <mailto:ad at mvsltd.com> ad at mvsltd.com 📧

Toll Free:  <tel:+18555786660> +1 (855) 578-6660📱️

 

Schedule A Meeting:  <https://calendly.com/aarondiblasi> https://calendly.com/aarondiblasi

News:  <https://mvsltd.com/news> https://mvsltd.com/news

Services:  <https://mvsltd.com/services> https://mvsltd.com/services

Testimonials:  <https://mvsltd.com/testimonials> https://mvsltd.com/testimonials

Facebook:  <https://mvsltd.com/facebook> https://mvsltd.com/facebook

X (Formerly Twitter):  <https://mvsltd.com/x> https://mvsltd.com/x

LinkedIn:  <https://mvsltd.com/linkedin> https://mvsltd.com/linkedin

Instagram:  <https://mvsltd.com/instagram> https://mvsltd.com/instagram

YouTube:  <https://mvsltd.com/youtube> https://mvsltd.com/youtube

Google:  <https://mvsltd.com/google> https://mvsltd.com/google


CONFIDENTIALITY NOTICE: This e-mail and attachments, if any, may contain confidential information, which is privileged and protected from disclosure by Federal and State confidentiality laws, rules, and regulations. This e-mail and attachments, if any, are intended for the designated addressee only. If you are not the designated addressee, you are hereby notified that any disclosure, copying, or distribution of this e-mail and its attachments, if any, may be unlawful and may subject you to legal consequences. If you have received this e-mail and attachments in error, please delete the e-mail and its attachments from your computer.

 

From: athen-list <athen-list-bounces at mailman12.u.washington.edu> On Behalf Of Joshua Hori via athen-list
Sent: Friday, April 11, 2025 4:56 PM
To: shelleyhaven techpotential.net <shelleyhaven at techpotential.net>; Karen McCall <K4mccall at outlook.com>; Access Technology Higher ATHEN <athen-list at u.washington.edu>
Subject: Re: [Athen] Handwriting to OCR text

 

Thanks for providing this explanation, Shelley! I appreciate the detailed comparison between handwriting OCR and dynamic handwriting recognition.

 

I just tested it out in OneNote, you can’t lasso handwriting in images. If an image is taken of a whiteboard with handwriting, it will not work. If you rewrite the handwritten notes, you can lasso that. 

 

Best,

Joshua

 

From: shelleyhaven techpotential.net <shelleyhaven at techpotential.net <mailto:shelleyhaven at techpotential.net> >
Date: Friday, April 11, 2025 at 1:01 PM
To: Karen McCall <K4mccall at outlook.com <mailto:K4mccall at outlook.com> >, Access Technology Higher ATHEN <athen-list at u.washington.edu <mailto:athen-list at u.washington.edu> >
Cc: Joshua Hori <jhori at ucdavis.edu <mailto:jhori at ucdavis.edu> >, Karlen Communications <info at karlencommunications.com <mailto:info at karlencommunications.com> >, Travis Roth <travis at travisroth.com <mailto:travis at travisroth.com> >
Subject: Re: [Athen] Handwriting to OCR text

I've been following this handwriting OCR thread with great interest.

 

While not a direct answer to Lorraine's original question (converting a PDF of handwritten notes to OCR'd text), here's one other thing to consider relevant to accommodations: there's a difference between handwriting OCR and dynamic handwriting recognition.

 

Handwriting OCR examines the static, after-the-fact image of handwritten text or math and attempts to recognize what the user wrote by analyzing the shapes and making a "best guess" by comparing these shapes with known handwriting samples.  The process is similar to the standard OCR we know and (mostly) love.

 

Dynamic handwriting recognition, however, analyzes handwriting in real time (as it's being written) and as such has access to additional information with which to make decisions: the individual strokes used to create the handwriting, the sequence of the strokes, the direction of the writing utensil, the velocity of the writing utensil, pen-up and pen-down movements, and possibly the pressure of the strokes.  This produces far more accurate handwriting recognition because it's analyzing "how the image was made", not just "what the image looks like".

 

Dynamic HWR obviously requires a system to capture the handwriting in real time, like a digitizing tablet or touchscreen.  OneNote on a touchscreen, Livescribe smartpens, Scribble on iPad with Apple Pencil, and many other apps & devices use dynamic HWR.

 

So in terms of accurately recognizing and converting a student's handwriting to text or math, it is advantageous to employ an accommodation that allows the student to capture their handwriting in real time using a dynamic HWR system.

 

- Shelley

 

_____________________________
Shelley Haven  ATP, RET, BSME
Assistive Technology Consultant
www.TechPotential.net <http://www.TechPotential.net> 

 

 

 

On Apr 11, 2025, at 5:53 AM, Karen McCall via athen-list <athen-list at u.washington.edu <mailto:athen-list at u.washington.edu> > wrote:

 

Yes.

 

I had to charge my Surface Pen so used the touch screen to “write” a sentence. I then used the Lasso tool to select the text I’d written.

 

I tapped on one of the handles of the lasso “square” and got the toolbar. There is an arrow in the toolbar which I tapped. The last item in the context menu is Ink to Text.

 

It worked perfectly. Even with my handwriting.

 

Once my Surface pen is charged, I’ll try it with that.

 

The way the handwriting tool used to work in OneNote is based on algorithms of thousands of handwriting sample, so it is sort of like word prediction based on the database of handwriting samples. Of course, like voice recognition, it also helps you write better/more legibly as you see the common “oops, I got that wrong” results if any. So, it tends to train you instead of the other way around.

 

Cheers, Karen

 

From: athen-list < <mailto:athen-list-bounces at mailman12.u.washington.edu> athen-list-bounces at mailman12.u.washington.edu> On Behalf Of Joshua Hori via athen-list
Sent: Thursday, April 10, 2025 8:13 PM
To: Karlen Communications < <mailto:info at karlencommunications.com> info at karlencommunications.com>; 'Access Technology Higher Education Network' < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>; 'Travis Roth' < <mailto:travis at travisroth.com> travis at travisroth.com>
Subject: Re: [Athen] Handwriting to OCR text

 

I’ve tried it and couldn’t get it to work, so I abandoned the idea for a while. I then realized later that you need to use the lasso to convert the handwriting to text.

 

So…Did the lasso convert imaged handwriting to text?

 

Joshua

 

 

From: Karlen Communications < <mailto:info at karlencommunications.com> info at karlencommunications.com>
Date: Wednesday, April 9, 2025 at 5:40 AM
To: Joshua Hori < <mailto:jhori at ucdavis.edu> jhori at ucdavis.edu>, 'Access Technology Higher Education Network' < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>, 'Travis Roth' < <mailto:travis at travisroth.com> travis at travisroth.com>
Subject: RE: [Athen] Handwriting to OCR text

Have you checked OneNote’s ability to convert handwriting to text? Not sure it would work on a scanned graphic of both handwriting and print but might be worth a try.

 

I used to use it all the time for note taking. Even with my horrid handwriting, it converted accurately.

 

…and you’ve just given me a task to test this morning!

 

Cheers, Karen

 

From: athen-list < <mailto:athen-list-bounces at mailman12.u.washington.edu> athen-list-bounces at mailman12.u.washington.edu> On Behalf Of Joshua Hori via athen-list
Sent: Tuesday, April 8, 2025 8:59 PM
To: Travis Roth < <mailto:travis at travisroth.com> travis at travisroth.com>; Access Technology Higher Education Network < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>
Subject: Re: [Athen] Handwriting to OCR text

 

The camera fails and throws errors now but if you hover over it, it will still try to OCR and read it aloud. We used to use SeeingAI back in 2018 to convert handwritten notes into digital text. It worked with good accuracy back then. 

 

This is the Image Braille GPT (free):  <https://chatgpt.com/g/g-fXlS5yyJA-image-braille> https://chatgpt.com/g/g-fXlS5yyJA-image-braille

 

It’s a little challenging navigating OpenAI with a screenreader since they don’t have ARIA live tags. I find Gemini more accessible to screenreaders. NotebookLM is accessible…with exception to navigating their chat responses. 

 

Best, 

 

Joshua Hori

Accessible Technology Coordinator

Information Educational Technology

Academic Technology Services

50 Hutchison Dr.

Davis, CA 95616

530-752-2439 

 <https://calendly.com/d/ytt-hsj-vbn> Schedule a meeting via Calendly

 

 

 

From: athen-list < <mailto:athen-list-bounces at mailman12.u.washington.edu> athen-list-bounces at mailman12.u.washington.edu> on behalf of Travis Roth via athen-list < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>
Date: Tuesday, April 8, 2025 at 5:34 PM
To: 'Access Technology Higher Education Network' < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>
Subject: Re: [Athen] Handwriting to OCR text

Hi Joshua,

On the Read tab SeeingAI does have a take Picture button which will still show the document view. But I’ve not tried handwriting with it, so if you are saying even the take picture mode is bad at handwriting that is unfortunate.

Can you share how I may find/try the Image Braille agent?

 

 

From: athen-list < <mailto:athen-list-bounces at mailman12.u.washington.edu> athen-list-bounces at mailman12.u.washington.edu> On Behalf Of Joshua Hori via athen-list
Sent: Tuesday, April 8, 2025 7:18 PM
To:  <mailto:foreigntype at gmail.com> foreigntype at gmail.com; Access Technology Higher Education Network < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>; <mailto:lorraine at rcn.com> lorraine at rcn.com
Cc: Access Technology Higher Education Network < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>
Subject: Re: [Athen] Handwriting to OCR text

 

SeeingAI used to convert handwriting to text, but now it just attempts to read it aloud…and it’s not as good as it used to be. That’s a shame.

 

You can upload an image of the handwritten text to AI and have it converted to digital text.

 

Gemini: Best AI for image descriptions and handwritten text.

ChatGPT: Creates great images but can go wildly inaccurate unless using the “Image Braille” GPT. That’s some high-quality system prompting powering that customized GPT.

 

I have video examples of using Gemini and OpenAI on my mobile device to describe lecture content in a LinkedIn article:  <https://www.linkedin.com/pulse/ai-musings-bridging-tactile-chemistry-lecture-audio-joshua-hori-kyglc/?trackingId=sKmgtHt1SPWYrhAnngJuVA%3D%3D> https://www.linkedin.com/pulse/ai-musings-bridging-tactile-chemistry-lecture-audio-joshua-hori-kyglc/

 

Sorry, no video or audio editing tools available. I stream/record on a budget.

 

Best, 

 

Joshua Hori

Accessible Technology Coordinator

Information Educational Technology

Academic Technology Services

50 Hutchison Dr.

Davis, CA 95616

530-752-2439 

 <https://calendly.com/d/ytt-hsj-vbn> Schedule a meeting via Calendly

 

 

 

 

From: athen-list < <mailto:athen-list-bounces at mailman12.u.washington.edu> athen-list-bounces at mailman12.u.washington.edu> on behalf of <mailto:foreigntype at gmail.com> foreigntype at gmail.com via athen-list < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>
Date: Tuesday, April 8, 2025 at 8:37 AM
To:  <mailto:lorraine at rcn.com> lorraine at rcn.com < <mailto:lorraine at rcn.com> lorraine at rcn.com>
Cc: Access Technology Higher Education Network < <mailto:athen-list at u.washington.edu> athen-list at u.washington.edu>
Subject: Re: [Athen] Handwriting to OCR text

I am unaware of any OCR software that can do both handwriting and text.

Susan's advice is the best: You'll have to do this in several steps. Use a good text based OCR software (Abbyy FineReader or OmnipagePro) for the text and insert (type up the handwriting manually) separately after removing it (by cleaning up) from the original.

No easy button for this one!

Wink

 

On Tue, Apr 8, 2025, 7:20 AM < <mailto:lorraine at rcn.com> lorraine at rcn.com> wrote:

Wink,

It is handwriting on a PDF.  It is the other software that I am looking for recommendations on.

Thanks

Lorraine

_______________________________________________
athen-list mailing list
 <mailto:athen-list at mailman12.u.washington.edu> athen-list at mailman12.u.washington.edu
 <http://mailman12.u.washington.edu/mailman/listinfo/athen-list> http://mailman12.u.washington.edu/mailman/listinfo/athen-list

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman12.u.washington.edu/pipermail/athen-list/attachments/20250411/8b205d4f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 94349 bytes
Desc: not available
URL: <http://mailman12.u.washington.edu/pipermail/athen-list/attachments/20250411/8b205d4f/attachment.png>