Lawyers for a New Jersey man charged are challenging the use of an increasingly common tool that has transformed DNA analysis in dozens of labs across the United States, saying the technique hasn’t been properly vetted for use in criminal courts.
A weekslong pretrial hearing about STRmix, which allows forensic analysts to test DNA samples that most likely would have been considered unusable a decade ago because they were too complex or small, ended this month in a Monmouth County courtroom in the case of Paul Caneiro, .
While Caneiro’s defense lawyers and experts have argued that the software hasn’t been proved reliable in the same way “safety-critical” systems used in cars and airplanes are — and that it could produce false results that could help wrongfully convict someone — prosecutors have argued that STRmix has been tested and tried in labs and courts across the country.
The “motivation is to actually test the software well, try and break it if we can, and, if we miss something, just honestly report what has happened,” one of STRmix’s developers, John Buckleton, testified last month, according to a transcript.
“I don’t want to contribute to an injustice ever,” he added.
A judge is expected to weigh in on the matter in February. Regardless of the outcome, said Marc Canellas, a public defender in Maryland who specializes in forensics and has handled cases that involve STRmix, the case highlights the long-standing need for stricter rules in an industry that can have accreditation standards but isn’t overseen by a regulatory authority.
“Even if this judge says that STRmix is completely unreliable, another judge in the same courthouse the next day could say that it is reliable,” he said.
Brutal deaths and then two house fires
Caneiro, who was 51 at the time of the killings, was charged with four counts of first-degree murder and other crimes in the deaths of his brother, Keith Caneiro, 50; Keith’s wife, Jennifer Caneiro, 45; and their two children, Jesse, 11, and Sophia, 8.
The New Jersey family was found dead at its home on Nov. 20, 2018, in Colts Neck, 47 miles south of New York City. Prosecutors have alleged that Caneiro fatally shot his brother, stabbed his niece and nephew and shot and stabbed his sister-in-law before he set their home on fire.
He then set his own house ablaze to destroy evidence and make it appear as if the entire family had been targeted, authorities have alleged.
Caneiro has pleaded not guilty. His previous defense team said there was “absolutely no reason in the world for Paul Caneiro to have committed the crimes he is alleged to have committed. He would never hurt any member of his family.”
Jury selection is expected to start in March.
Among the evidence cited by prosecutors were more than a dozen DNA samples that analysts processed using STRmix. STRmix’s technology promises to help solve what has long been an issue in DNA analysis: The tinier the sample and the more complex it is — if it contained genetic material from multiple people, for instance — the more difficult it has been to properly analyze, said William Thompson, a professor emeritus of criminology, law and society at the University of California, Irvine, who has studied DNA for decades.
Probabilistic genotyping
Traditional DNA analysis was largely limited to samples that contained genetic material from a couple of people, said Monica Ghannam, a forensic scientist who testified in the Caneiro hearing, .
Using STRmix, she testified, analysts can evaluate mixtures with DNA from four people, the newspaper reported.
Developed by scientists in New Zealand and Australia, the technology is one of a handful of forensic tools that use “probabilistic genotyping” to try to remedy that problem, said Jack Ballantyne, a professor of chemistry at the University of Central Florida and associate director of research at the National Center for Forensic Science.
The software uses statistical modeling to analyze complex mixtures of genetic material that may have been obtained from something as small as a few human cells left on a doorknob, Thompson said.
The makers of the technology say that STRmix can “de-convolute” such mixtures to identify the genetic profiles of the people who may have left those cells behind, Thompson said, and that the software can estimate how much more likely it would be to find those profiles on the doorknob if the DNA came from a person of interest and not a random person.
STRmix is the most widely used software in the United States that claims to do that, Ballantyne said.
“It has revolutionized the ability to analyze complex mixtures,” he said. “There’s no question that the U.S. is moving the DNA community lock, stock and barrel into the probabilistic genotyping arena.”
Whether public or private, he added, the majority of labs in the United States that do forensic casework involving DNA most likely use the method.
New technology, new results
One set of results obtained by the New Jersey State Police DNA lab illustrates the possibility of the technology. A pair of jeans found in Paul Caneiro’s basement had a bloodstain on the shin area, and when analysts examined it in 2018 using traditional DNA methods, they found that it contained genetic material from two people, said Christine Schlenker, a forensic scientist in the lab, a transcript of her testimony shows.
But the quality of the DNA wasn’t good enough to yield a result, Schlenker said. After the lab obtained the software, an analysis using STRmix showed that the blood in the stain appeared to mostly be that of Paul Caneiro’s nephew, Jesse, Schlenker said. The DNA was 2.7 septillion times more likely to have come from Jesse than someone else, she said.
Monmouth County prosecutors wouldn’t comment on STRmix or its uses in the Caneiro case. At the pretrial hearing, prosecutor Nicole Wallace said STRmix has been tested repeatedly and is generally accepted in the scientific community that tracks and evaluates such technology, according to a transcript. And in a brief filed this year, prosecutors said there have been court hearings across the United States evaluating what defense lawyers have argued is “novel” software.
“The results have been consistent,” the brief says. “STRmix is not new — it is tried and tested.”
But Caneiro’s lawyers and software engineers who testified for the defense said that testing hasn’t been carried out adequately or independently. Such testing has been key to preventing catastrophic software failures in aviation systems and nuclear weapons, said Canellas, the Maryland public defender, as well as on devices with far less serious stakes.
“We do this type of testing for mobile games and mobile apps, and yet we aren’t doing it for the criminal legal system,” he said. “That, to me, is just appalling.”
The world’s largest engineering organization, the Institute for Electrical and Electronics Engineers, has previously called on developers of digital forensics tools — including probabilistic genotyping software — to follow its strictest standard for independent verification and validation to avoid “false imprisonment and the deprivation of people’s rights,” the group said in a statement three years ago.
That standard, which has been used on nuclear weapons and space exploration probes, requires independent testing at three levels — technical, financial and managerial — and it aims to assess everything from a product’s fundamental concept to accurate coding and the likelihood of false results, the statement said.
In his closing argument, Caneiro’s lawyer, Christopher Godin of the New Jersey Office of the Public Defender, said the people behind STRmix are “intimately” involved in helping DNA labs set up and test it.
Buckleton, the STRmix developer, testified that the people who work on the software assist labs in their validations they assist labs in their validations and offered an analogy: “If you had a new Pratt & Whitney jet engine for your Boeing aircraft, would you like to be handed the manual and say go for it?”
Buckleton, who is principal scientist at the New Zealand Institute of Environmental Science and Research, described the standard from the IEEE, as the engineering organization is known, as “sensible” and “desirable” and said STRmix conforms with it.
But Canellas, who has a Ph.D. in aerospace engineering and is a member of IEEE, said he has seen not publicly released details corroborating Buckleton’s statement. (A manager for STRmix said it wouldn’t comment on the matter.)
‘Too much ambiguity’
Caneiro’s lawyers also pointed to several DNA analyses in the case that appeared to violate the labs’ internal validation studies, or testing that labs do before they start using the software to establish what samples they can reliably analyze.
The samples relied on less DNA than what had been examined in those studies, according to the defense brief, and the labs didn’t test for a key part of the samples in the Caneiro case — “relatedness.”
Relatives share genetic markers, the brief says, and interpreting their DNA can be far more challenging. The Caneiro case involves five family members: four victims and one alleged perpetrator.
“There’s too much ambiguity,” Godin said in his closing argument.
Wallace, the prosecutor, countered that the analysts who examined the Caneiro samples had no difficulty assessing the relatives’ DNA and that neither did experts who reviewed them later for the prosecution. Wallace acknowledged that one of the labs analyzed samples that were below what they’d examined in their validation studies, but she pointed to a recent appellate ruling that described that not as a fundamental flaw in the forensics, but as an issue for juries to grapple with.
Thompson, of UC Irvine, said that from what he has seen, STRmix works well if it’s used under the conditions it has been tested for. But, he said, labs can run into trouble when they analyze complex types of samples that haven’t been validated, especially if they’re relying on tiny amounts of DNA.
“I’ve been following the development of DNA technology for over 30 years now,” said Thompson, who wasn’t familiar with the Caneiro case. “There’s a long history of people getting enamored of the technology and taking it a little bit too far and not quite understanding what they’re doing. And I think that could definitely happen with regard to probabilistic genotyping. Probably it already has happened.”
