75 Years of Innovation: Handwriting recognition

75-years-of-innovation-handwriting-recognition-feat-img

The 75 Years of Innovation series highlights the groundbreaking innovations spanning from SRI’s founding in 1946 to today. Each week, SRI will release an innovation, leading up to its 75th anniversary in November 2021.

75-years-of-innovation-handwriting-recognition-feat-img

The ability to verify handwritten signatures in real-time


The stroke of a pen and a verifiable signature

There have been many paths leading to our current digital society. One of these paths has been the long and winding road to transforming handwriting into something a computer can read. Developing the human-computer interface is nothing new to SRI International; after all, SRI experts such as Douglas Engelbart invented the computer mouse.

In the 1960s, SRI worked on solving another outstanding issue — making handwriting machine-readable. This would solve complex issues such as signature forgery and help with the input of east Asian language characters into computer systems. The result was the SRI Pen, a mechanism that could capture handwriting in real-time.

Putting pen to digital paper

SRI has a history of working out solutions to elusive problems, especially those in which people and computers connect. In the late 1950s, SRI had worked on the electronic recording machine and accounting (ERMA) system, and as part of that project, had developed magnetic ink character recognition (MICR). This project piqued the interest of SRI’s Hew Crane; while the ERMA project worked on getting computers to read printed characters, Crane wanted to work out a way that a computer could read handwriting. This meant that the act of writing, in which a pen impacts a writing surface, could be the instrumentation through which data is collected, processed, and understood.

handwriting recognition

Hew Crane using the Kanji handwriting recognition pen

This logical leap turned out to be the key to automatic handwriting verification and was the first step toward behavioral biometrics. ERMA was the inspiration for Crane, who began the project looking at the writing of checks. While ERMA only processed printed characters, Crane realized that the dynamic use of the pen was as much a part of a person’s signature as the signature itself. This behavior is used in today’s science of behavioral biometrics to detect fraud in account login and other online activities.

The first SRI Pen was demonstrated in the 1960s and was based on a direction-sensitive pen programmed to recognize the movements required to write the digits 0 to 9. The pen sensed four directions used in the sequence of writing digits. However, this proof of concept, like many demonstration systems, had unacceptable limitations; the pen had to be held in the prescribed orientation, and the same movement sequence had to be used to write the numbers. The concept, however, was proven.

How automatic handwriting verification helped write off forgers

Like many innovative leaps, it took time for this idea to mature. In 1975, SRI was granted a patent for a handwriting system, described here:

A system for identifying handwritten characters is provided wherein a pen and associated circuitry generate a sequence of signals representing a sequence of directions which is taken to write each character.

By the mid-1970s, with increasing interest from industry, SRI built a new version of the SRI Pen that used an array of strain gauges to measure writing force in three dimensions. Also, around that time, Xebec Systems sponsored further exploration of the SRI Pen to recognize all alphanumeric characters. Continuing development led to better flexibility in character recognition and the reduction of restrictions for left-handed writers. These improvements resulted in the development of the pen as a signature verification system. The increasing use of credit cards provided a need for this type of biometric recognition system, particularly for point-of-sale (POS) terminals. VISA and other credit card companies became customers, but the SRI Pen was negatively impacted by the economics of the credit card world. The signature verification equipment’s cost versus the false rejection rate was deemed to be too high when combined with the vast majority of charges being for small amounts.

Demonstrating-the-Kanji-pen-for-CBSs-Dan-Rather-in-April-1976-Handwriting-recognition

Demonstrating the Kanji pen for CBS’s Dan Rather in April 1976

Research continued under the sponsorship of the Defense Nuclear Agency. In 1981, SRI conducted a year-long study to evaluate the effectiveness of automatic signature verification based on three-axis signature dynamics. During the study, 5,220 signatures and 1,740 numeric sequences were collected from 59 subjects. In addition, twelve trained forgers attempted 648 forgeries. The forgers used true signers’ signatures, and each was trained in how the SRI signature verification system worked. They were even shown a video of the true signers writing their signatures and were allowed to practice over three-weeks.

An equal error rate (EER) is a term used in biometrics to describe a biometric system’s accuracy. An EER is where the FAR (false acceptance rate) and FRR (false rejection rate) intersect. A device with a low EER is considered to be more accurate. The study results found that dynamic signature verification based on a three-axis pen system had equal-error rates on the order of 1% using a standard set of features for all subjects.

The digital pen writes itself into history

Because of the SRI Pen’s potential, including its ability to capture east-Asian characters, the invention continued to garner interest. In 1981, an SRI spin-off company, Communications Intelligence Corporation (CIC), was formed. CIC introduced products such as a pen-input notebook computer from NCR Corporation and the MacHandwriter, a product based on Japanese writing and distributed by Apple.

In the 1990s, personal digital assistants (PDAs) and some cellphones were equipped with a stylus or digital pen. Certain laptops also were given the power of the pen. These pens provided a way to capture handwritten text and convert it to a machine-readable form. SRI’s innovative automatic signature verification set the path for the behavioral biometric technology of today.

It took a vision and decades of innovation to turn something we take for granted, handwriting, into a powerful digital medium. The power of the pen persists. iSIGN (formerly CIC) is an industry leader in e-signature solutions, offering simple, click-to-sign web browser signatures to multimillion-dollar, highly customized implementations.

Resources

The Dish, 75 Years of Innovation: The computer mouse: https://medium.com/dish/75-years-of-innovation-the-computer-mouse-fef5161ba45d

The Dish, ERMA, A Banking Baby Boomer: How SRI Helped the Bank of America to Become the First Automated Bank:
https://medium.com/dish/75-years-of-innovation-banking-automation-erma-f297ad8c55fd

U.S. Patent 3,930,229, Crane, et.al. 1975

1981 study published by SRI International and sponsored by the Defense Nuclear Agency into automatic signature verification effectiveness: https://apps.dtic.mil/sti/pdfs/ADA111329.pdf

CIC (iSIGN): https://isignnow.com/


Read more from SRI