Biometrics in the 21st century: Computers cannot replace humans yet
Novaator visited the Estonian Forensic Science Institute (EKEI) to learn about the work of forensic experts. In the biometrics department, they observed how fingerprints and facial images are collected and processed. Discussions also touched on what kind of person is well-suited to comparing dozens of fingerprints and facial images day after day.
If fingerprints are left at a crime scene or a perpetrator is captured on, for example, a security camera image, this data is processed by the biometrics department. According to Andra Sirgmets, head of the department, their work focuses on two areas: fingerprints, considered a "historical classic," and facial recognition, a new capability still in its implementation phase. "People mistakenly think that comparing faces is easy. In reality, those who work in both fields have said that making decisions based on faces is more complicated than with fingerprints," Sirgmets explains.
Gunnar Tasa, deputy director for development at EKEI, emphasizes that for comparisons to be effective, at least one of the two images being compared must be of high quality. This is especially important given that the first round of comparisons is now typically performed by a computer. Unfortunately, fingerprints found at crime scenes are often partial, and security camera footage can be blurry. "This highlights the need to collect high-quality reference materials. If the image obtained from the crime scene is poor, then at least the comparison image must have features that are easy for the computer to detect," Tasa explains.
Who left these prints...
The main building of the Forensic Science Institute in Tallinn houses one of Estonia's 26 collection sites. In other words, the facility includes a room where a suspect can provide fingerprints and have a high-quality facial image taken. "Similar setups or equipment can be found in Police and Border Guard Board (PPA) police stations and in three prisons," explains Andra Sirgmets.
Depending on the severity of the crime, all suspects are required to provide a DNA sample, a facial image and fingerprints. Fingerprints are collected far more thoroughly than during a document application process. Both hands' fingerprints and thumbprints are scanned, including both press and rolled impressions. Before fingerprints taken from a suspect can be compared with those found at a crime scene, the latter must first be made visible, according to Sirgmets. "If an offense has been committed and we receive an object of interest, the fingerprints on it are generally not visible," the expert explains.
Fingerprints are made visible through a process called development. According to Sirgmets, this involves various physical and chemical methods, such as a cyanoacrylate (superglue) fume chamber or powders. "Often, a single method isn't sufficient, so different treatments are applied sequentially, depending on the surface and material of the object," she notes. Once the fingerprints are made visible, experts photograph them and perform image processing.
Next comes the analysis and comparison with an individual, Sirgmets explains. "The traditional approach is to use a magnifying glass and examine the prints, but a more modern method involves specialized software," she says. The prints are entered into ABIS (Automated Biometric Identification System), where comparable features can be marked manually or with computer assistance.
"ABIS is essentially an image bank containing pictures of fingerprints and faces," explains Gunnar Tasa. It also functions as a search engine capable of comparing face to face and fingerprint to fingerprint. Currently, ABIS contains around 2,000 facial images, while the fingerprint database holds fingerprints from over 160,000 individuals collected during previous investigations.
The process begins with an expert conducting a search in the database. "We get a list of candidates from the search, and the expert compares the trace of interest with them one by one," Sirgmets describes. Even if there is no actual match, the system provides a list of potential candidates. It is then up to the expert to manually verify whether any of the suggested matches truly correspond to the suspect's trace. "The idea that the computer instantly provides a perfect match along with the perpetrator's home address, like in movies, doesn't happen," Sirgmets clarifies.
When comparing fingerprints, the expert examines three levels of features. First, the general pattern of the fingerprint is assessed, but the actual comparison, for both humans and algorithms, focuses on the unique characteristics of the dermal ridge patterns. "Then there are third-level features, which include extremely fine details, such as skin pores," Sirgmets explains.
According to Tasa, technology has not yet advanced to the point where human involvement in fingerprint comparisons can be eliminated. "Today, achieving an automatic comparison of ten fingerprints is already considered quite good," he acknowledges. This means that if two complete sets of ten fingerprints could be compared perfectly, the system might be trusted. "However, if you have only a single trace from a crime scene, full automation is definitely not feasible yet," Tasa concludes.
Faces the most misleading
While fingerprints have been studied and stored in Estonia for decades, facial recognition only became possible on September 1, 2023, with the enactment of a legal amendment. Since then, facial images of suspects have gradually been added to ABIS. "Before this, our country didn't have a facial database that could be effectively searched during criminal proceedings," recalls Andra Sirgmets.
Although facial recognition is commonly used to unlock smart devices, its application in forensic science is a completely different matter, according to Sirgmets. Experts often work with low-quality photos or videos, which may be the only trace left behind by a perpetrator. "The suspect's face is cropped from the frame, and the system extracts specific features – called feature vectors – from the facial image to create a biometric template," Sirgmets explains.
A biometric template is essentially a dataset that describes the face. "While we, as humans, perceive similarity with our eyes, the algorithm compares data to data," the expert adds. Similar to fingerprints, ABIS generates a list of candidates for an expert to review after a facial search. Once again, the actual perpetrator may or may not be among the suggested candidates. Given the current state of technology, a human expert must manually compare each facial candidate individually.
"When it comes to fingerprints and DNA, we can talk about matches, but with faces, we cannot, as facial recognition is a weaker biometric modality," Sirgmets clarifies. In other words, facial images are the most challenging to compare. Factors like camera angles, lighting, as well as a person's weight, age and other variables all come into play. Despite these challenges, experts follow a strict protocol to review all candidates and present their opinion to the court on whether, for example, the individual captured on a security camera is among the suspects.
According to Gunnar Tasa, EKEI is nearing the completion of recruiting natural talents for the biometrics field. Candidates with no prior training are shown pairs of fingerprints and faces to compare over the course of a day, and their ability to make decisions for hours on end is evaluated. Experts agree that comparing fingerprints and faces requires immense perseverance and concentration. "Our goal is to find the small percentage of people who are naturally so skilled that, with additional training, they will be ready to perform effectively in the role," Sirgmets explains.
When it comes to facial recognition, Estonians are at a relative disadvantage due to the country's small population and its predominantly fair-haired, Nordic demographic. "It's said that if you come from a small tribal village and have seen only ten people in your life, your ability to recognize faces is very underdeveloped," Tasa reflects. In contrast, someone who grew up riding the New York City subway has encountered a wide variety of races and faces. "For us, the most challenging faces to compare are those of Asian and African descent," Tasa admits.
Facial recognition is still in its early stages of implementation in Estonia. This process was preceded by several years of groundwork through the international TELEFI project. "We mapped the state of facial recognition searches and databases across Europe: what has been done and where progress has been made," Tasa recalls. The project provided EKEI with ideas on how to structure its own database. "We certainly aim to develop it to the point where the database reaches a reasonable size and we can start offering a new service to our investigators," Sirgmets affirms.
--
Follow ERR News on Facebook and Twitter and never miss an update!
Editor: Marcus Turovski