1 You will now read a transcript of testimony given by Terry Smith at trial.

Q: Please state your name.
A: Terry Smith.

Q: Who do you work for?
A: The local police department.

Q: What do you do for the police department?
A: I am a firearms examiner.

Q: How long have you been doing that?
A: 7 years

Q: What is a firearms examiner?
A: A firearms examiner is someone who looks at cartridge cases and bullets to determine whether they were fired from a particular firearm.

Q: What training is required to become a firearms examiner with the local police department?
A: I received my bachelor's degree in forensic science and in 2015 I transferred to the crime lab from the crime scene unit. I underwent a two-year training program, which was supervised by experienced firearms examiners; I've toured manufacturing facilities and saw how firearms and ammunition were produced; and I've attended several national and regional meetings of firearms examiners.

Q: And during the course of your career, were you tested in proficiency to make sure you were still conducting appropriate examinations?
A: Yes. I have undergone annual proficiency examinations.

Q: Is the local police department lab accredited?
A: Yes, it is accredited by ASCLID/LAB.
2 Q: And in addition to what you just told us, do you have other qualifications?
A: Yes, I do.

Q: Can you tell us what those are?
A: Yes. I received training in the use of a bullet matching algorithm. This is an algorithm that evaluates the characteristics of two fired bullets, in order to produce a score for the similarity of the bullets, where more similar bullets are more likely to have been fired from the same gun. I attended a workshop on the algorithm on 1/11/2020, held by CSAFE - Center for Statistics and Applications in Forensic Evidence. The training taught me to use the algorithm alongside my personal judgement. I found that my conclusion was reflected in the similarity score produced by the algorithm in all 21 cases.

Q: Where does the bullet matching algorithm come from?
A: It was created by Dr. Adrian Jones.

Q: How long have the state police been using the bullet matching algorithm?
A: They have been using it since January of 2020.

Q: Have you testified in court previously using the bullet matching algorithm?
A: Yes, I have, approximately 10 times.

Q: As a firearms examiner, have you testified about your conclusions, given the results of your testing?
A: Yes, I have.
3 Prosecution: Your Honor, at this time I would ask that Terry Smith be qualified as an expert in the field of firearms identification subject to cross examination.
Court: Any cross on their credentials?
Defense: No, Your Honor.
Court: This witness is an expert in the area of firearms identification. They can testify to their opinions as well as facts. Go ahead.

Q: What work did you do on this case?
A: I was asked to compare a bullet from the crime scene to a test fire from the gun recovered from the traffic stop.

Q: Did you examine how many lands and grooves the bullet had?
A: Yes.

Q: Can you explain for the jury what that means?
A: Yes. In the interior of a barrel there are raised portions called lands and depressed areas called grooves. When a bullet passes down the barrel, a bullet will spin and that gives it stability and accuracy over a distance. Those raised areas are designed by the manufacturer. They're cut into the barrel. And each particular file has a different combination of lands and grooves. But essentially what those lands do is grip a bullet and spin it, and as that bullet passes down the barrel, it scratches the random imperfections of that barrel into the bullet.

Q: Now, for these bullets, you counted up the lands and grooves and determined the direction of the twist, correct?
A: Yes. This bullet had six lands. And the interior of the barrel, the barrel will either twist right or it will twist left. And in this particular case, the barrel twists right. And you can see that by looking at the bullet. If you look at the base of the bullet, either it goes to the left or goes to the right.
4 A: Exhibit A depicts a cross-section of a barrel, where rifling is clearly present. Exhibit B depicts an example of a fired bullet, with which includes lands and grooves.
---Exhibit A---

rifling

---Exhibit B---

fired_bullet
5

---Test-fired bullets admitted into evidence---


Q: Can you describe the process of obtaining these test-fired bullets?
A: The test-fired bullets came from a test fire of the gun recovered from the traffic stop.

Q: You mentioned test firings; can you explain what that means?
A: In test firing, first what I would do is make sure the firearm is safe to actually test fire. Then I would use lab ammunition and I would test fire it, meaning that I am creating a fired bullet. Typically you do two at a time. That way you have a fired bullet to compare to another fired bullet.

Q: Would you then have taken the test firings you created and did you compare those test firings to the fired evidence that you had also received?
A: Yes. First what I would do is compare my test shot to test shot. I am looking for a detailed microscopic pattern. Once I have done that then I would compare it to the fired evidence.

Q: And how about the number of lands and direction of twist for the test fires?
A: It also had six lands, and twisted to the right.

Q: Okay. Now, did you compare the test-fired bullets to the fired evidence under the comparison microscope?
A: Yes, I did.

9 A: Exhibit C shows an image of a comparison microscope, as well an an example of a matching bullet comparison.
---Exhibit C---

microscope
10 Q: Did you use an algorithm to compare these bullets as well?
A: Yes, I used the algorithm to compare the two test fires to each other. I also used the algorithm to compare the better-marking test fire to the fired evidence that I received.

Q: Could you explain how this algorithm compares bullets?
A: The algorithm uses 3D measurements to make a comparison between the surface contours of each of the lands on each bullet. These comparisons result in a match score between 0 and 1, where 1 indicates a clear match, and 0 indicates that there is not a match. The bullet is aligned based on the maximum agreement between the lands, and the average match score for the lands is computed. This average score gives an overall match score for the entire bullet.

Q: What was the match score between the two test-fired bullets?
A: The match score was 0.976.
11 A: Exhibit D shows the comparison grid between the two test-fired bullets.
---Exhibit D---

test_fire

18 Q: Now, how many times have you compared bullets to determine if they were fired from the same gun?
A: I'd say thousands.

Q: And do you ever see two bullets that have agreement in every single area of the bullet?
A: No. When firing a firearm there is a dynamic process because it is a contained explosion. When the firing pin hits the primer, which is basically the initiator, what gets it going, it will explode, burn the gun powder inside the casing, and the bullet will travel down the barrel, picking up the microscopic imperfections of the barrel, and the cartridge case will slam rearward against the support mechanism. During that dynamic process, each time it happens, a bullet will be marked slightly differently from one to the next.

Q: When you reached a conclusion, did you write up a report?
A: Yes, I did.

Q: Is it the local police department's protocol to have somebody else who's a firearms tool mark examiner in your lab review that report, review your work, and determine if it's correct?
A: Yes.

Q:That's what we call peer review?
A: Peer review, yes.

Q: Thank you, no further questions.

22 Q: Is there something fixed about the amount of what has to be found to constitute sufficient agreement?
A: No, there is not a fixed amount or a numerical value.

26 Q: How long did you say you've been trained in the bullet matching algorithm?
A: Since January of 2020.

Q: Okay. So that's fairly new, is that fair to say?
A: It is still fairly new, yes.

Q: The software uses modeling; is that correct?
A: Yes, it does.

Q: You, personally, don't know the source code; is that correct?
A: That's correct.

Q: And, in fact, you, personally, would not be able to tell us the specific math that goes into this program; is that fair to say?
A: We did receive training on what the math is doing in general terms, but I am not a statistician, and would prefer to let them speak to that.

You will now read a transcript of testimony given by Adrian Jones at trial.

Q: Please state your name.
A: Adrian Jones.

Q: Who do you work for?
A: CSAFE, the Center for Statistics and Applications in Forensic Evidence.

Q: What is your current occupation?
A: I am currently a Professor of Statistics.

Q: How long have you been doing that?
A: 30 years.

Q: What are your qualifications with regards to the bullet matching algorithm?
A: I have a Ph.D. in Statistics, and I have spent 7 years developing the bullet matching algorithm. I have spent 8 years collaborating with firearms examiners during the development and rollout of this algorithm.

Q: Are you familiar with the bullet matching algorithm?
A: Yes. I was involved in the development of the algorithm.

Prosecution: Your Honor, at this time I would ask that Adrian Jones be qualified as an expert in the bullet matching algorithm, subject to cross examination.
Court: Any cross on their credentials?
Defense: No, Your Honor.
Court: This witness is an expert in the area of the bullet matching algorithm. They can testify to their opinions as well as facts.

Q: How many times have you testified regarding this bullet matching algorithm?
A: 17 times.

Q: Could you describe how this bullet matching algorithm compares bullets?
A: Yes. For certain types of guns, the barrel will have lands and grooves, known as rifling. This rifling spins the bullet in order to make its trajectory more stable. Due to the manufacturing process, this rifling can produce identifiable markings on the bullet, based on random differences between barrels. Because of these random imperfections, the striation marks left on bullets can be compared in order to determine if it is likely that they were fired from the same gun.

The first step is to determine where the lands on the bullet are located. These lands will be the sunken area that contains the striation marks between the smoother grooves. 3D scans are then taken for each land, and the “ shoulders ” , or area transitioning from the land to the grove, are excluded from the analysis.
A: Exhibit F shows the identification of a land.
---Exhibit F---

shoulder
Next, a stable area of the 3D scan containing the striations is selected, and a cross-section of this area is used to show the striations along with the topology of the region. A smoothing function is applied to remove some of the imaging noise from the 3D scan, leaving the striae intact. A second smooth is subtracted from the striations in order to remove the curvature of the region, leaving only the striae - this is what we call a signature.

A: The signature for the two bullets being compared are aligned such that the best fit between the two signatures is achieved. The striation marks between the two signatures are then compared by evaluating how many of the high points and low points correspond. The algorithm can calculate the number of consecutively matching striations (CMS), or consecutively matching high points and low points - these are features used directly by some examiners to characterize the strength of a match. It also calculates the cross correlation between the two signatures, which is a numerical measure of the similarity between the two lands ranging between -1 and 1.
A: Exhibit G shows two matching signatures, while Exhibit H shows two non-matching signatures.
---Exhibit G---

signature

---Exhibit H---

signature
These traits are combined using what is known as a random forest. Each forest is composed of decision trees, which use a subset of the observed values in order to make a decision about whether or not the bullets constitute a match. The other observations are held out in order to determine an error rate. When the random forest makes a prediction, each decision tree “ votes ” , producing a numerical value between 0 and 1 corresponding to the proportion of trees which evaluate the features as being sufficiently similar to have come from the same source.

Q: Have you tested this algorithm?
A: Yes. This algorithm was tested and validated on a number of different test sets of bullet scans. It was found that, as long as there are sufficient marks on the bullet, the algorithm could successfully distinguish between bullets fired by the same gun and those fired from different guns. Examiners' visual comparisons are also limited by the presence or absence of individualizing marks. Two test sets were using consecutively rifled barrels, which should be the most difficult to assess, and it was shown that the algorithm could distinguish between the bullets fired from two separate guns with complete accuracy.

Q: Can this algorithm be used on 9mm bullets fired from a Ruger LCP firearm, such as the firearm in question for this case?
A: Yes, this algorithm can be used on these types of bullets, given that this type of firearm marks well.

Q: Has this algorithm been published?
A: Yes. The algorithm and its process have been discussed in peer reviewed journals such as Law, Probability, and Risk , The Annals of Applied Statistics , and Forensic Science International . The algorithm is also open source, which means that the full source code, documentation, and numerical weights are available online for anyone to examine.
---Cross Examination---

Q: How long has the bullet matching algorithm been used in court cases?
A: Since January of 2020.

Q: Okay. So that's fairly new, is that fair to say?
A: It is still fairly new, yes.

Q: This algorithm requires some decision making on the part of the operator, such as how much the signature is smoothed and what part of the bullets are inputted into the system, correct?
A: Yes, there are certain parameters which must be specified, but the system defaults are usually sufficient and have been validated with a number of different firearms and ammunition types. There are also operating protocols for determining which parts of the bullet are scanned, so while this process is manual, there are clear criteria and the associated variability from scanning is well understood, and published in Forensic Science International .

Q: Can this algorithm be applied to all bullets?
A: No, the algorithm only works on traditionally rifled bullets which are largely undamaged. The algorithm has not been validated to work on seriously damaged or fragmented bullets.