Welcome to Crime Lab Report

* * * * *
Low-bid accreditation will doom forensic science.

December 20, 2007 by Crime Lab Report

Click here to view printer-friendly version of this month's editorial.

If crime laboratory directors don't recapture their vision for a unified and cohesive system of accreditation, the integrity and effectiveness of our country's criminal justice system will suffer horribly.

Twenty-five years of progress is being threatened by a dangerous fracture that has developed in the profession and is being pried open with bad information and the absence of community oversight.

The ending to this strange story has yet to be written. But to fully understand the significance and history of the issues in this matter, our readers will benefit from a look back at how forensic-science accreditation came about and why it must be saved from the clutches of a renegade philosophy that doesn't seem to concern itself with the long-term good of the profession.

The Birth of Forensic Science Accreditation
Forensic Science as we know it has been practiced in North America for nearly a century, but the application of today's more reliable quality-assurance strategies is a relatively new trend that came about thanks to accreditation.

In the early 1970s, prior to accreditation, disturbing problems in some of America's crime laboratories were exposed by a voluntary proficiency testing study conducted by the Forensic Science Foundation with support from the Law Enforcement Assistance Administration (LEAA). National news organizations soon caught word of the findings, which eventually made headlines throughout the country.

At about the same time, FBI Director Clarence Kelley, joined by FBI Laboratory Director Briggs White, was in the process of organizing a group of crime laboratory directors that eventually became known as the American Society of Crime Laboratory Directors, or ASCLD. One of ASCLD’s earliest committees was the Committee on Laboratory Evaluation and Standards. Its objective was to improve the tarnished image of the nation’s crime labs by developing quality assurance standards to which they could voluntarily demonstrate conformance through rigorous external scrutiny. The committee refined its recommendations based on input from the ASCLD membership. In 1981 the ASCLD / Laboratory Accreditation Board (ASCLD/LAB) was formed.

By 1984, the number of accredited laboratories reached the prescribed threshold required by the organization’s bylaws to form an assembly of delegates that could democratically govern the new accreditation program.

ASCLD/LAB was officially incorporated as a not-for-profit organization in 1988. As the number of accredited laboratories continued to grow, crime labs in Australia, Canada, Hong Kong, Malaysia, New Zealand, and Singapore would eventually join with their American counterparts in supporting the accreditation system that continues today.

The DNA Dilemma
The accreditation movement hit a road bump in 1994 with federal passage of The DNA Identification Act. The act established a powerful DNA Advisory Board (DAB) to develop and enforce quality assurance standards for crime laboratories wishing to access the national database of DNA profiles maintained by the FBI. The DAB recommended that crime labs seek accreditation “with all deliberate speed.”

Because ASCLD/LAB policies and procedures would not allow accreditation to be awarded to a single work unit, laboratories that were not prepared to undergo a full ASCLD/LAB accreditation assessment seemed to have no other alternative but to forfeit access to the DNA database until they were ready for a full accreditation audit.

In 1995, a new organization called the National Forensic Science Technology Center (NFSTC) was formed by the executive board of ASCLD with the approval and support of the ASCLD membership. The purpose of this private not-for-profit corporation, as defined in its Articles of Incorporation, was for training, education, certification, and the support of accreditation. In a letter to FBI Director Louis Freeh, Florida Congressman Bill Young urged that NFSTC “would fulfill a national need to disseminate forensic information, provide forensic training and education in the latest techniques and technologies, focus applied forensic research, and provide for the continued development and implementation of forensic standards.”

To this day, NFSTC continues to be a valuable resource for forensic science laboratories throughout the United States.

The creation of NFSTC, however, also presented an opportunity to reconcile the DNA dilemma. First, NFSTC could support and assist crime laboratories preparing for a full ASCLD/LAB accreditation inspection. Second, NFSTC could audit and temporarily certify DNA units that complied with DNA-specific quality assurance standards, thereby satisfying the requirements of the DAB. It seemed like the perfect solution to a nagging problem that many crime laboratories were trying desperately to resolve. It also provided one more opportunity for NFSTC to secure a future for itself by gaining favor with government crime laboratories in the U.S.

In order to alleviate concerns that the young and struggling NFSTC might be tempted to develop a competing accreditation program of its own, a memorandum of understanding was signed by NFSTC Executive Director Bill Tilstone and ASCLD/LAB Chair Jo Ann Given in 1997. In this agreement, NFSTC promised to abide by the following conditions:

1. NFSTC would not initiate the DNA certification process unless the applicant laboratory had first given a written commitment to seek full ASCLD/LAB accreditation within three years of the laboratory’s certification.

2. NFSTC would not re-inspect or re-issue a certification to any laboratory that had not formally applied for ASCLD/LAB accreditation within three years of the date of its first certification by NFSTC.

Clearly, this was a no-competition agreement that seemed to preserve the original vision of a strong and united system of accreditation.

Mixed Messages and Broken Promises
A peculiar thing happened in 2000 when NFSTC went forward with the development of an accreditation program for the Georgia Bureau of Investigation (GBI), which would be based on the new ISO 17025 international standard for testing and calibration laboratories. When ASCLD/LAB complained that this was an apparent breach of their agreement, NFSTC President Michael Sheppo argued in a letter to ASCLD/LAB that “NFSTC would not have responded to the GBI if ASCLD/LAB were ISO 17025 compliant and will not offer ISO 17025 accreditation to any public crime laboratory in the U.S. once the ASCLD/LAB program achieves that standing.”

To further complicate the understanding of NFSTC's business motives, NFSTC announced in 2001 that part of its lucrative agreement with the National Institute of Justice (NIJ) would be to assist laboratories seeking to earn accreditation through the ASCLD/LAB program.

Astonishingly, with the GBI project slated to move forward, NFSTC had now committed itself to both supporting and competing against ASCLD/LAB. Why the National Institute of Justice allowed itself to be a party to such an obvious conflict may warrant scrutiny of its own. But for now it is clear that NFSTC and NIJ were leaving many to scratch their heads in wonder of how these events would all play out.

They had their answer on January 1, 2004 when NFSTC spun-off its accreditation services to form a new independent accreditation corporation called Forensic Quality Services, or FQS. FQS had previously operated as a separate business unit within NFSTC so as to divorce NFSTC's federal funding from its cost-recovery activities. In a decision that some observers argued was a blatant disregard of previous agreements, NFSTC had created a direct competitor to ASCLD/LAB and a second accreditation service in the forensic science community.

FQS was quick to rationalize the tactics of its parent organization by claiming that it was not NFSTC but ASCLD/LAB and its delayed entry into the ISO accreditation market that created the competition. In fact, FQS now bills itself as the "country's longest established provider of ISO accreditation to forensic science testing laboratories in the U.S."

The facts, however, paint a more accurate picture.

The majority of nationally recognized accreditation programs in the sciences and other professional arenas are generally developed, governed, and operated by authoritative professional communities. ASCLD/LAB's deliberate and slower transition to ISO 17025 began well before FQS entered the market, allowing the pace to be dictated by the forensic-science management community. By keeping the community together, earning its stamp of approval, and leveraging the power of its collective wisdom, the ASCLD/LAB International program now carries a degree of legitimacy that is more in keeping with the original concept of a unified accreditation system.

ASCLD/LAB has now accredited 55 laboratories under its International program in addition to the 277 laboratories accredited under the original Legacy program.

FQS-International, on the other hand, has accredited just over 50 laboratories. But since FQS allows forensic laboratories to "customize" their accreditation by phasing in one discipline at a time, many of their client labs are only accredited in one or two disciplines.

The Detroit Police Department, for example, is listed by FQS as one of its accredited laboratories. But when Crime Lab Report studied Detroit's scope document, we learned that the laboratory is only accredited in one work unit: biology. What about ballistics? What about fingerprints? Should any lab be called accredited when not all of its primary disciplines are included in the scope of the assessment? We don't think so.

A Dangerous and Intolerable Situation
Crime Lab Report believes that marketplace competition will eventually destroy forensic science accreditation. By definition, no two professional services are the same. So the very existence of multiple accreditation programs invites the reasonable assumption that one must be better than the others. And if the community of forensic science managers is to be entrusted with the governance of such a critical profession, why would inferior programs be allowed to exist at all? Don't the safety of the public and the rights of defendants deserve only the best?

Of course they do. And only the best they should have.

Unfortunately, the community of crime laboratory directors has been largely deprived of the opportunity to deliberate and speak authoritatively about the repercussions of multiple accrediting bodies in forensic science. And to make matters worse, accreditation decisions are now more likely to be influenced and disrupted by apathetic procurement officials who argue that the existence of competition obligates crime laboratories to publicly solicit bids for accreditation.

This cannot be tolerated.

Forcing accrediting bodies to compete on the basis of up-front cost estimates creates a dangerous conflict of interest by tempting bidders to scale-down the scope and/or intensity of an inspection in order to maximize the recovery of costs or avoid monetary losses. Until an accreditation process begins, there is simply no way to project how much time or manpower will be needed to properly complete the project. So how can an accrediting body be expected to submit a responsive bid?

FQS has insisted that “best practices in public procurement would seem to require that a fair, competitive bidding process be used to determine the 'best value' provider of accreditation for a public crime laboratory.” FQS also wrote in its March 2007 newsletter that it "supports the general principle that fair and open competition results in driving quality up and costs down, and that the forensic science community benefits as bidders work to meet the requirements of the customer in the most cost-effective way that they can.”

Wait a second. Would the work-quality of two competing auto-mechanics improve if they were forced to give estimates without having the opportunity to examine the cars first?

Absolutely not. Costs would rise as both mechanics inflate their estimates to protect themselves from unforeseen complications. Then, quality would decrease when the less competitive mechanic realizes he can stay afloat a bit longer by cutting a few corners and using cheaper parts. Sure, market forces may eventually drive consumers away from the bad mechanic, but by that point the damage has been done.

If we were talking about the purchase of a car, Crime Lab Report would agree with FQS's position on competition in the marketplace, but we're not. We are talking about a critical professional service where the qualifications, reputation, and community-backing of the provider can have a direct impact on the effectiveness and reliability of our criminal justice system.

ASCLD/LAB clearly recognized this when it made the following announcement in May of 2007:

“Laboratories should be free to select an accrediting body that best serves their needs....You may be assured that if a laboratory makes it clear which accrediting body they have selected, ASCLD/LAB will not interfere by making appeals to higher authorities within the agency. Selecting an accrediting body is a most important decision for you and your laboratory and you should not be subjected to outside interference.”

Accreditation Must be Owned by the Community
Years ago, the community of crime laboratory directors designed and implemented a unified program of accreditation that allows participants to seek election to an executive board of directors that includes a diversity of talent and perspectives. Through this democratic and competitive process, new ideas and improved standards can be brought forward for consideration and incorporation into the program.

- continued on the right

Letters to the Editors

MARYLAND FINGERPRINT RULING: Our coverage of the controversial fingerprint ruling in Maryland drew several interesting comments. Here are just a few.
------------------------------------
Jay Siegel, Ph.D., Director of the Forensic and Investigative Science Program, Indiana University / Purdue University at Indianapolis

Crime Lab Report misses the point of the judge’s ruling. The point is that fingerprint examinations are entirely subjective. The underlying scientific principle, that all fingerprints are measurably, demonstrably different is unproven and no amount of case work will prove it. Case work is not set up to test this principle. The FBI’s problem in the Mayfield case was not error rates, it was more culture and observer bias. Each successive examiner knew what the previous examiner had found. No one that I know suggests that DNA probabilities are error rates.

You are correct that fingerprint “science” has a PR problem. When the FBI sends in an examiner who states that fingerprint ID is infallible and there is 0% error rate, that is a problem. I think that the judge’s opinion was reasonable given the evidence she had. The fingerprint community must get away from the idea that no errors are ever made with fingerprint IDs and start doing the research that would verify (or not) the underlying principle. Research must continue that would make this analysis less subjective.
-------------------------------------
Patrick N. Collier
UAS Security Products Corp

As has been pointed out, nothing is ever perfect in nature. Errors are seen all of the time.

Another problem to be considered is the media's approach to things that are attention-getting without any study or consideration for the impact of their reporting. They prosper by bringing to the public attention any question of impropriety on the part of law enforcement.
-------------------------------------
Ray Wickenheiser, Consultant

Forensic Consulting, Irvine, TX

Thank-you for a well reasoned editorial and very pertinent background information surrounding the Mayfield and Rose cases. Even the most perfect science, when placed in human hands, must be practiced with care. Accreditation and all of the quality assurance checks and balances that come with it are forensic scientists' best response when faced with the "throw the baby out with the bathwater" approach posited by the defense. The favored approach by forensic scientists should include some humility in even the most straightforward examinations. While the science is sound, care and attention must be practiced to ensure the outcome is correct.
-------------------------------------

Forensic Science in the News

Georgia
December 9, 2007
A bipartisan study panel for the Georgia House of Representatives has proposed legislation, which if passed, would require minimum procedures for improving the accuracy of witness identifications.

Washington
December 8, 2007
The Yakima Police Department will open a new state-of-the-art crime laboratory thanks to a sales tax increase. The new laboratory will be approximately five times larger than the existing facility.

Texas
December 7, 2007
The Texas Chapter for the Innocence Project will be assisting lawyers in the review of 180 serology cases worked by the embattled Houston PD Crime Laboratory. Students will be assigned to provide assistance during the review.

New York
December 4, 2007
The Inspector General for the state of New York has issued a report that closes a seven-month investigation into allegations that chemists reported results for drug cases that were not analyzed, or not sufficiently analyzed - a practice called dry-labbing. A former NYPD lab director faces possible criminal charges for allegedly covering-up the scandal. The state's Inspector General has recommended that the Queens District Attorney file charges against W. Mark Dale, who is no longer employed by NYPD.

Colorado
December 2, 2007
Members of a task force created by Governor Bill Ritter announced that it will recommend new legislation to establish evidence collection and preservation procedures for biological evidence in major felony crimes.

Turner Entertainment
November 17, 2007
A new book titled Beating the Devil's Game showcases the history of forensic science. Author Katherine Ramsland, Ph.D., explores the inside world of this fascinating profession in an uncomplicated and clear style that is easy to understand.

Washington DC
November 17, 2007
A new crime laboratory for the District of Columbia is scheduled to open in 2011. The total project cost is expected to reach $219 million. Mayor Adrian Fenty will ask the federal government for $31million to close a funding gap.

Seattle, WA
November 9, 2007
Prosecutors have declined to charge Ann Marie Gordon, a toxicology lab manager, for falsely certifying quality-assurance samples used in DUI tests. Gordon resigned her position in July.

Low-bid accreditation (continued):

The claim that marketplace competition is necessary for improvement and should trump the collective wisdom and guidance of the very community that gave birth to accreditation is irresponsible.

We also take issue with the supporters of competition that have argued that the relatively new international ISO 17025 standard is an equalizer that relegates accreditation to a kind of commodity that is indistinguishable from provider to provider.

This is incorrect for two important reasons.

First, accreditation is not a commodity; it is a lifeline that adds a layer of security to protect the public and its justice institutions from sloppy science and misplaced methods. Second, ISO 17025 is not a standard that is specific to forensic science. It is for evaluating the operations of all kinds of testing and calibration laboratories throughout the world. Without supplemental standards to "amplify" it, ISO 17025 would fail to encompass all of the critical functions that crime laboratories monitor on a daily basis.

Accreditation Must be Enforced by the Community
Crime Lab Report is genuinely troubled by how the forensic science community has been forced to accept competing accreditation programs without having the opportunity to weigh in on it. In science, the community rules. Steam-roller tactics that alienate the community should be dealt with by leaders who are bound to protect and preserve that community. Much more could have been done - and should have been done - to prevent a small group of individuals from unilaterally altering the landscape of forensic science.

We also reject the notion that crime laboratories should be allowed to seek "partial" accreditation or take advantage of their procurement procedures to negotiate customized terms of accreditation that were never approved by a consensus of the forensic-science-management community. Parameters such as the length of the accreditation cycle, the duration of the assessment, the number of assessors, and the manner in which the assessment is conducted should never be negotiated. They should be standardized.

Cafeteria-style accreditation is not accreditation. If important standards, policies, and practices are allowed to flex at the pleasure of each crime laboratory seeking accreditation, there will be nothing left for the forensic science community to present as proof that it is capable of governing itself. Then, it will only be a matter of time before legislatures impose their own brand of bureaucratic control.

A true and effective accreditation program forces crime laboratories to subordinate themselves to the standards and scrutiny of the forensic science community - not the other way around. Egotists in the profession who cringe at the thought of subjecting themselves to this kind of "indignity" should either join and converse with their more fair-minded colleagues or find another line of work.

An accreditation assessment must always be a thorough and rigorous examination of a laboratory's entire operation, and it must be conducted under the watch and authority of a united professional community. This is the only way to ensure that accreditation is awarded by the most qualified, reputable, and reliable body available.

ASCLD/LAB is the singular accreditation authority conceived, established, and governed by the forensic science management community in the United States. There is no other. Its purpose is to continually improve the quality of forensic science and must be vigilantly protected. Conflicts of interest and subversive tactics that bypass the scrutiny of such a highly respected group of professionals will quickly erode the confidence that people have in the ability of accreditation to govern the profession.

Settling for less is not just a bad value - it is bad business.

Please click here if you would like to respond to this commentary. We welcome opposing viewpoints.
--------------------------------------------------------------------------------------------------

EDITORIAL NOTE:
ACCREDITATION PROCUREMENT TIPS

If any crime laboratory finds itself being harassed in its attempt to make a sole-source accreditation selection, we would like to know about it immediately.

Our extensive research for this month's editorial revealed that it is highly appropriate for crime labs to utilize a sole-source selection process when preparing for accreditation. By definition, no two professional services are the same. As long as a crime laboratory is able to document and explain this effectively for its procurement representatives, there should be no fall-out from the process.

In the Federal Acquisition Regulation, FAR 6.302, seven exceptions can be used to justify a sole-source procurement:

1.   There is only one responsible source and no other services will satisfy agency requirements
2.   There is an unusual or compelling urgency
3.   The vendor provides certain engineering or expert services
4.   The selection is related to an international agreement
5.   The product or service is required by statute
6.   The selection is in the interest of national security
7.   The selection is in the public interest

Exceptions 1, 3, and 7 apply to forensic science accreditation and can be used easily to justify sole-source selection. But if a laboratory finds itself being forced by its parent agency to publicly solicit bids or proposals for accreditation services, Crime Lab Report urges laboratory administrators to utilize the increasingly popular Qualification-Based Selection (QBS) process.

What is Qualification Based Selection?
QBS was originally established by the Congress of the United States to evaluate engineering and architectural services for federally funded construction projects. A growing number of states continue to embrace this concept and apply it to many professional services where quality and value are critical. Courts have also ruled that when certain professional or scientific credentials are needed to produce a desired level of quality, QBS is superior to the traditional bidding process. During the QBS process, accrediting bodies should not be asked to submit cost estimates due the conflict of interest that this creates. Evaluate reputation and qualifications; and remember, the process is not supposed to be easy. Nothing worthwhile ever is.

Wishing you safe and happy holidays.

Our editorial board would like to wish our readers a very safe and happy holiday season. We are grateful for the overwhelming support and encouragement that we have received over the last several months. Our goal is to continue to improve our service so that our readers can stay informed and engaged. We look forward to more success in 2008.

Best Wishes!