CSA: Three years in

The Federal Motor Carrier Safety Administration is responsible for enforcing federal safety regulations for more than half a million truck and bus carriers. In an effort to focus its compliance resources on carriers that pose the greatest threat to safety, FMCSA launched the Compliance, Safety, Accountability initiative in late 2010. CSA’s Safety Measurement System rates carriers’ risk of a future crash based on how many violations of federal safety regulations they receive during roadside inspections and crash investigations.

More than three years after its launch, recently released reports have come to conflicting conclusions about whether CSA is achieving its intended goals. An FMCSA-funded report released Feb. 5 determined that the program is more effective at identifying at-risk carriers than FMCSA’s previous program, SafeStat, which ran from 1997 to 2010. Meanwhile, an audit released the same week by the Government Accountability Office noted that CSA has allowed FMCSA to better focus its compliance resources, but questioned whether CSA’s Safety Measurement System assigns safety scores fairly and tracks violations that actually correlate to future crashes.

“Just because CSA is an improvement over previous programs does not make it a ‘good’ program,” Bill Graves, president and CEO of the Arlington, VA-based American Trucking Associations, said in a press release. He agrees with GAO’s assessment that “scores produced by the program don’t present an accurate or precise assessment of the safety of many carriers.”

Changes to compliance reviews

Under SafeStat, FMCSA’s only tool for addressing unsafe carriers was a comprehensive onsite compliance review. These reviews were “labor intensive,” according to GAO, and resulted in FMCSA conducting reviews on a small percentage of carriers.

With the new system, FMCSA now has more tools and flexibility to identify a carrier’s actual safety deficiencies, GAO noted in the report. FMCSA’s investigators can now conduct focused onsite or off-site reviews based on the carrier’s actual violations. For example, an investigator can skip an onsite review for recordkeeping violations and instead ask the carrier to submit its records digitally for review.

CSA also automatically generates warning letters for carriers after they receive a set number of violations in a particular safety performance category. The letter indicates what the safety problem is and allows the carrier an opportunity to challenge its score. If the carrier does not improve, FMCSA can take additional actions.

Chuck DeWeese, assistant commissioner for the New York Governor’s Traffic Safety Committee, said these changes allow FMCSA to intervene with more carriers.

“Why go through a carrier’s logbook records if hours of service is not their problem and maintenance is?” he said. “Instead, just do a focused review of their maintenance section. It’s operating smarter with fewer resources.”

In its audit, GAO found that between fiscal years 2007 and 2012, FMCSA increased its number of annual safety interventions to about 44,000 from about 16,000, largely with the help of the warning letters.

For many small truck carriers, safety may be low on their list of priorities, DeWeese said. The warning letters and other resources FMCSA provides alert these smaller carriers that they are putting themselves at risk for a crash in the future, he said.

CSA’s effect on operations

CSA’s Safety Measurement System rates carriers in seven Behavior Analysis and Safety Improvement Categories, which include compliance with hours-of-service regulations and proper maintenance of vehicles. The BASICs attempt to focus on a broader range of behaviors linked to crashes. SafeStat focused on a small number of serious violations that result in out-of-service orders, while the BASICs track more than 750 possible violations of federal regulations.

The sheer number of violations that the Safety Measurement System tracks now allows carriers to have more insight into how their safety programs can be improved, according to Charles J. Schwarting, vice president of Summerfield, FL-based National Safety Consulting Services Inc.

Under typical circumstances, the Safety Measurement System assigns carriers scores from 0 to 100 in the BASICs based on the frequency and severity of violations and crashes compared to other similar carriers, with 100 representing the worst safety performance. When carriers reach a certain score, they are flagged as high risk and become candidates for CSA’s intervention process, which typically begins with a warning letter. If improvements are not made, it could lead to an investigation – and if violations continue, FMCSA could place the carrier out of service.

Interventions such as the warning letters have increased employers’ focus on making safety a shared responsibility among all employees, said Dennis L. Shinault, director of Loss Prevention for Indianapolis-based Baldwin & Lyons Inc., which provides marketing and underwriting insurance for the transportation industry. For instance, Shinault notes that maintenance crews at some companies with which he has consulted are conducting more in-depth inspections before each trip to make sure the driver will not receive a vehicle maintenance violation during a roadside inspection.

The FMCSA-funded report concluded that carriers flagged as high risk in the Safety Measurement System had double the national crash-rate average, and nearly 4 out of 5 of the carriers had higher crash rates than carriers that would not have been flagged in the SafeStat program.

In addition, an annual survey conducted by the American Transportation Research Institute, the research arm of the American Trucking Associations, found that more than half of the drivers surveyed in 2012 and 2013 reported CSA has caused them to conduct more pre-trip and post-trip safety inspections.

The Safety Measurement System methodology

Although some stakeholders believe CSA has led to positive safety changes among carriers, others have criticized the BASIC scores as unfair and uncorrelated to future crash risk.

In its audit, GAO looked closely at CSA’s methodology for calculating BASIC scores and how well the scores correlated with a carrier having a future crash. GAO found that only a small number of violations that CSA monitors, such as speeding, are significantly associated with having an actual crash in the future. Others, such as having an expired medical examiner’s certificate, are counted against carriers’ scores but are not closely associated with having a future crash.

GAO also found that most carriers had a small sample of inspection data. The more inspections a carrier receives, the more precise the scores will be; however, small carriers that operate fewer than four vehicles – which made up two-thirds of GAO’s sample population in the audit – will naturally have fewer inspections.

“For example, for a carrier with five inspections, a single additional violation could increase that carrier’s violation rate 20 times more than it would for a carrier with 100 inspections,” GAO noted.

The office concluded that it is likely “SMS scores do not represent an accurate or precise safety assessment for a carrier” and do not effectively determine which carriers have a higher risk of a future crash.

“Basing a carrier’s safety fitness determination on limited performance data may misrepresent the safety status of carriers, particularly those without sufficient data from which to reliably draw such a conclusion,” GAO’s report states.

FMCSA’s journey to improvement

GAO recommended that FMCSA be more selective in the carriers it scores based on which ones have the most relevant safety data. Although this will result in fewer carriers with ratings, the office stated, the scores likely will be more reliable and allow FMCSA to better allocate its limited resources. The Department of Transportation agreed to consider GAO’s recommendations.

In an audit report released March 5, the DOT Office of Inspector General also made a series of recommendations for FMCSA to improve CSA after finding that the Safety Measurement System contained inaccurate and incomplete data, among other issues. In response, FMCSA Administrator Anne S. Ferro pledged to improve the systems’ data quality and make other recommended improvements, such as providing states with sufficient resources to implement CSA enforcement interventions.

“FMCSA has always said that CSA is, and probably will be for the foreseeable future, a work in progress,” Schwarting said. “They are doing the best they can do to continually make improvements … and I expect that they are going to get better and better.”