NASTC's Comments on Proposed Apnea Rulemaking
April 26, 2016
Docket Services
U.S. Department of Transportation
Room W12–140
1200 New Jersey Avenue, S.E.
Washington, DC 20590–0001
RE: Docket No. FMCSA-2015-0001,
Carrier Safety Fitness Determination, Proposed Rule
To Whom It May Concern:
The National Association of Small Trucking Companies (NASTC), which represents more than 6,000 commercial motor carriers, is pleased to provide comments on the Notice of Proposed Rulemaking regarding Safety Fitness Determinations (SFDs), published January 21, 2016 (FMCSA-2015-0001).
NASTC is a member-based organization whose members range from a single or two or three power units to more than 100 power units; however, our members average 16 power units. These companies operate in the long-haul, over-theroad, full-truckload, for-hire sector of interstate trucking. NASTC’s members come from the largest segment of America’s long-haul trucking — small motor carrier businesses. Our members and we are committed to safety.
We commend the agency that the proposed rule attempts to address some of the most egregious aspects of CSA and its deleterious effects and bias against small trucking companies. These include little to no inspection data for a given carrier, wide fluctuation in BASICs scores, and significant state enforcement variation.
We begin our comments with the proposed use of these BASICs, the framework on which the proposed SFD is based. Next, we make comment on several elements of the proposal, including the number of inspections and the appeals process, offering suggestions for improving the rule. Finally, we comment on the process for this rulemaking.
BASICs and SFDs
The Compliance Safety Accountability (CSA) program has relied upon categories called BASICs (Behavior Analysis and Safety Improvement Categories), using data including that from roadside inspections, and SMS methodology, to purport to rate motor carrier safety. These same BASICs and SMS remain at the heart of the proposed Safety Fitness Determination rule. Yet, SMS and BASICs have been found wanting. They contain inadequate, inconsistent, incomplete data; they lack valid methods for calculating and draw invalid, unreliable conclusions about carriers’ actual safety performance and future crash risk. Yet, the proposed rule rests upon the same widely criticized SMS and BASICs.
Congressional oversight hearings in 2012 by the House Small Business Committee and the House Transportation and Infrastructure Committee illuminated the fundamental flaws that characterize CSA’s Safety Measurement System (SMS). This is important to note because the SFD proposed rule would build on the foundation of the SMS now ordered to be reconstituted.
Research entered into the hearing records, which included analysis by Professor James Gimpel of the University of Maryland and reports by Wells Fargo, detailed how SMS’s use of sparse inspection data is misleading; it overstates the crash risk of motor carriers with no or small samples of inspections and understates the crash risk of motor carriers with far larger numbers of inspections. Rated compliance data only correlate meaningfully for the largest carriers having significant numbers of inspections. That is, SMS measurements lack statistically significant correlation to crash predictability for as much as 90 percent of carriers, and inspections and enforcement actions vary considerably among states, creating what Wells Fargo called a “negative feedback loop” regarding a given carrier’s purported safety performance.
Subsequently, separate examinations by the Government Accountability Office and the Department of Transportation’s Inspector General arrived at the same conclusions as Wells Fargo, the University of Maryland, and others: SMS and BASICs lack accuracy when matched to real-world carrier safety performance, namely due to lack of sufficient inspections on which to base valid assessments.
Notably, since CSA went into effect and made its carrier scores public in December 2010, independent operators and small trucking firms such as NASTC’s members have been isproportionately, adversely affected by the flawed Compliance Safety Accountability program’s Safety Measurement System and its misleading BASICs. CSA has caused objectively safe small carriers to lose business, some even to go out of business, because this government program branded them as “unsafe” despite the objective truth, as judged by such sound measures as accidents per million miles driven. And while the proposed rule cites DataQ as a means of improving safety data quality, the DataQ process puts the enforcement officer who issued a motor carrier a citation in position to quash any appeal. This unfair, biased procedure was brought to light in the congressional oversight hearings and the DOT IG’s report mentioned above.
The agency is to be commended for acknowledging the distortions arising pursuant to the “law of small numbers” and the adverse effect of relative, as opposed to absolute, values of performance data and ratings. Whereas small motor carriers have suffered the most under CSA to date, this acknowledgement and turn toward use of absolute performance measures are welcome.
However, the continued use of SMS and BASICs for proposed SFDs, given the problems that have plagued them, does not provide a satisfactory foundation for basing Safety Fitness Determinations. The proposed rule describes as Unfit Method 1 (section VI(C)) a “path” to a “proposed unfit” determination. It rests squarely on indicators from BASICs. The BASICs are not reconstituted under the proposed rule. They and SMS would remain as they have existed. The “absolute” performance basis employed to proceed to making an SFD, under the proposed rule, would still rely upon spotty data and flawed methodology. The same disconnection between actual carrier safety performance and false positives arising under SMS and BASICs would apply under Unfit Methods 2 and 3, insofar as these paths each depend on unreconstructed SMS and BASICs. In all three proposed paths to SFD, two failed BASICs would result in a carrier being “proposed unfit.” This is a serious thing, given its questionable basis and the serious disruption, risk, and harm to a motor carrier. By streamlining to a single SFD status under this proposed rule, the stakes are heightened and the remediation possibilities are reduced.
Therefore, NASTC urges the agency to forego usage of SMS and BASICs, as currently constituted, as central elements for initiating a Safety Fitness Determination. Alternatively, we recommend use of valid on-road safety performance indicators, namely a threshold number (related to the number of a motor carrier’s power units) of serious crashes (within a 2-year time frame) where the truck driver is determined to be at fault and the accident results in loss of human life, serious human injury, or significant property damage. Accidents per million miles driven, a recognized, objective measure of a carrier’s safety performance, should be used in combination with actual crash frequency, severity, and fault. We appreciate that the proposed rule incorporates such measures in adjusting under the Unsafe Driving category.
Threshold Number of Inspections
Section VI(A) states that, as proposed, the initiation of a Safety Fitness Determination in many instances would be based on a carrier’s having “at least 11 inspections in the previous 24 months.” This number of inspections proposed seems arbitrary. It is a questionable threshold figure. It bears no relation to the number of months, such as average of one inspection per every two months of the 2-year period (which would be 12 inspections) or an average of one inspection per month of the 2-year period (which would be 24 inspections).
It is statistically suspect to set such a low threshold of inspections to trigger an SFD. The Government Accountability Office’s analysis of SMS methodology and BASICs, derived from inspection data, found a dearth of inspections for many carriers, especially small carriers. GAO recommended a minimum of 20 inspections in order to have enough data on which to derive valid statistics. The 20-sample figure for a minimum is not arbitrarily chosen. Rather, 20 is broadly regarded as the minimum acceptable among researchers. “A great many experienced researchers use a rule-of-thumb sample size of approximately twenty. Smaller samples often result in low-power values . . . .” As another research methods expert notes, “First, statistics calculated from large samples are more accurate, other things equal, than those calculated from small samples.”
GAO determined that reliance on small samples of inspection data introduces statistical bias into SMS, BASICs, and CSA and, therefore, makes for an unacceptable basis for concluding carrier lack of safety. For example, GAO said, "For most BASICs, we found FMCSA’s data sufficiency standards too low to ensure reliable comparisons across carriers. In other words, many carriers’ violations rates are based on an insufficient number of observations to be comparable to other carriers in calculating an accurate safety score."
Further, GAO said, "For the five SMS BASICs for which FMCSA uses relevant inspections as a measure of exposure — Hours-of-Service Compliance, Driver Fitness, Controlled Substances and Alcohol, Vehicle Maintenance, and Hazardous Materials — estimated violation rates can change by a large amount for carriers with few inspections even when the number of their violations changes by a small amount. For example, for a carrier with 5 inspections, a single additional violation could increase that carrier’s violation rate 20 times more than it would for a carrier with 100 inspections. This sensitivity can result in artificially high or low estimated violation rates that are potentially imprecise for carriers with few inspections."
Finding carriers that have had 20 inspections within a 24-month period may be problematic for the agency, but the statistical and real-world consequences of a recommended unfit SFD matter. It should not be the goal to conduct SFDs for their own sake. Rather, the integrity of the data and methodology, along with cognizance and mitigation of the harm potentially inflicted on small carriers, on freight-hauling capacity, and of other economic consequences should be a main concern. Less than 20 inspections in a 2-year time is an unacceptably low threshold number of inspections.
Single SFD “Unfit” Rating
The proposal in Section VI(B) would move from the current safety fitness ratings of satisfactory, conditional, and unsatisfactory to a single determination of “unfit.” While streamlining SFD categories might have some benefit, the proposal actually leaves two categories: “unfit” following completion of a “proposed unfit” process and “unrated,” though the latter category is not named or discussed in the proposed rule. It would be more reasonable to acknowledge that the agency has not performed an SFD on a carrier and it, therefore, is “unrated,” and to clarify that no corresponding “fit” category exists and that any motor carrier that is authorized, licensed, and insured is regarded as fit to operate. Alternatively, a clarification could say that a carrier’s being unrated is the equivalent of being authorized, licensed, and insured and thus deemed fit to operate.
Safety Event Groups and Failure Rates
The agency is to be credited for its recognition of and efforts to remedy the adverse effect on small motor carriers from “the law of small numbers,” as seen in failure rate of safety event groups raised in Section VI(C). To an extent, applying different failure standards within different BASICs would ameliorate the distortions that harm small carriers. That is a good first step.
While the baseball analogy is helpful, it does not go far enough to fit the situation of the use of BASICs and safety event groups as it relates to small carriers. Just as the statistical weaknesses discussed above regarding small samples show, and just as the number of few at-bats relate to a baseball player’s having an incredibly high, unsustainable or incredibly low batting average in the first week of the season, small carriers with under 20 inspections in two years should not be targeted for an SFD on SMS-based safety event “failure.”
NASTC agrees with the need for adjustment for the many thousands of small carriers having few inspections. However, the proposed reliance on SMS and BASICs, fewer than 20 inspections in 24 months, and the disconnection from actual at-fault accidents of sufficient severity would set too low a standard. The high, varied percentages with each BASIC, as proposed, are necessary but not sufficient. These would still have the effect of throwing carriers having mere form-and-manner, paperwork, state enforcement variation, etc. troubles under SMS’s flaws into SFDs.
Therefore, for small motor carriers, any “safety event” and “failure rate” should be directly and strongly connected to serious crashes where the truck driver is at fault; and that resulted in human death, serious injury, or significant property damage. Such a step would respond to GAO’s finding: "FMCSA’s method of categorizing the carriers into safety event groups for the remaining BASICs also demonstrates how imprecision disproportionately affects small carriers." Further, GAO concluded: "Overall, these results raise concerns about the effectiveness of the existing SMS as a tool to help FMCSA prioritize intervention resources to most effectively reduce crashes." A Safety Fitness Determination is the ultimate intervention. Thus, the trigger that launches such an intervention must not be a hair-trigger. And the steps for assessing carrier safety fitness must be much more dependent upon actual crash and severity than is proposed.
SFD Appeals Process
Section VII describes the proposed process for appealing SFDs that result in an “unfit” determination. First, we are concerned by the extraordinarily tight deadlines proposed for appealing an “unfit” determination. Allowing just 10 days for a carrier to file an appeal regarding unconsidered inspection data and 15 days to request administrative review of material errors are extremely short deadlines. This is especially so, compared with the present 90-day deadline afforded motor carriers seeking due process to remain in operation. Motor carriers would still have contractual obligations to fulfill, extreme weather or other extenuating circumstances to contend with, a company owner may need the counsel or assistance of others in preparing such appeals, or other valid reasons may prevent a carrier from meeting the proposed short deadlines. These seem unreasonably burdensome for small carriers, who lack the large administrative and business executive staff resources necessary timely to meet the deadlines.
Second, subsection (C), relating to appeals to continue to operate under a compliance agreement, states that carriers granted such relief would have the fact of operating under a compliance agreement disclosed publicly on FMCSA’s website as long as the agreement was in force. Publicizing operation under a compliance agreement would seem to revive the scarlet-letter-type branding that proved so harmful under CSA before the FAST Act required the agency to stop publicizing CSA ratings.
Further, a compliance agreement, as proposed, would indicate a carrier’s commitment to improvement. Thus, the carrier would be objectively in greater compliance and steadily improving from the moment the agreement took effect. That motor carrier would be branded as having suspect safety performance as it was during its lowest ebb; yet, that carrier would in all likelihood be demonstrably more in compliance and safer in operations thereafter. Nevertheless, the agency’s branding such carriers on its website would very likely cause adverse economic effects, just as objectively safe but CSA-branded carriers experienced. Loss of business at such a time would prove highly disruptive and destabilizing, and would exacerbate the situation for carriers trying to do the right thing. NASTC urges that compliance agreements not be publicly disclosed.
Process
NASTC is troubled by the process and timing the SFD proposed rulemaking is employing. On December 3, 2015, Congress enacted the Fixing America’s Surface Transportation (FAST) Act. This highway authorization law called for fundamental reforms to CSA, including reformulating and correcting its methodological and data flaws that underlie SMS and BASICs. The FAST Act lays out a process that will take a significant period of time, at least a couple of years, for laying a new, corrected foundation on which CSA would operate — and which will serve as the basis for the proposed SFD rule. This process involves analysis by the National Academy of Sciences, accountability to the DOT Inspector General for correcting CSA pursuant to the FAST Act, and certification by the Inspector General that the data are sufficient, consistent, and accurate to ensure validity and reliability.
Less than two months later, on January 21, 2016, FMCSA proceeded with publication of this notice of proposed rulemaking. It is obvious from the timing and from the rulemaking itself that the agency was well underway toward proceeding with this rulemaking, and Congress’s clear directives in the new law would not stop this regulatory action. It would have been prudent and appropriate to postpone publishing this rulemaking, once Congress sent such a clear signal — writing a new law, in fact, that explicitly mandates that the agency send the foundation on which this rule builds back to the drawing board.
Further, this rulemaking ignores Section 5202 of the FAST Act, which directs the agency to use only an advance notice of proposed rulemaking or a negotiated rulemaking for any major rule. It is plain that delineating a new procedure for Safety Fitness Determinations is a major rule.
Moreover, this proposed SFD rule abrogates Section 5221 of the FAST Act, as the proposed rule clearly “relates to the CSA program, including the SMS or data analysis under the SMS.”
Then to fast-track this rulemaking through a 60-day comment period on an NPRM, rather than allow at least 90 days, compounds the perception and the effect of foregoing an ANPRM. This is not the proper way to proceed with such a significant measure with such far-reaching consequences.
Thus, for a proposal that continues to embrace the vestiges of the very CSA house of cards the Congress recently mandated FMCSA to overhaul, particularly with respect to its methodology and data for purporting to gauge a motor carrier’s safety performance, causes NASTC and its members grave concern. This proposed rule would perpetuate some of the most scrutinized, invalidated aspects of a “safety” enforcement regime that is now beginning to undergo massive overhaul. Yet, that intransigent commitment to a failed experiment is what this proposed rule represents, and it does so in a manner that disrespects the rule of law and the will of Congress.
* * *
In conclusion, NASTC appreciates FMCSA’s interest in the input of those in the trucking industry who will have to live under any new Safety Fitness Determination rule. We appreciate the opportunity to offer our comments, and we urge the agency to reconsider the issuance of this rulemaking. Despite the efforts represented in the rulemaking as proposed, its design could be interpreted as resting upon an assumption that smaller motor carriers are generally less safe and that larger carriers are generally safer. However, the experience illustrated by a certain large carrier versus that of a comparatively smaller carrier shows that such an assumption is not well founded. Despite the larger carrier’s adoption of electronic on-board recorders and having the benefit of many inspections on which to base CSA scores, its CSA composite figures, its accidents per power unit, and fatalities per power unit indicated safety performance less safe than the smaller carrier, by a factor of three to four times. More important to actual safety would be to assess driver retention rates, dispatcher-driver ratios, accidents per million miles per power unit, and accident severity, when assessing a carrier’s safety fitness.
Furthermore, given the lack of data on the vast majority of small motor carriers, the approach proposed for moving toward assignment of “unfit” status would amount to a “close down” order for many a small company. The due process and something akin to presumption of innocence strike us as lacking. Thus, the fate of many small carriers seems to rest upon the same faulty data and methodology that have been their bane under CSA, only the consequences are all the graver under this NPRM. There must be far more substantial due process afforded these small carriers, if the agency is going to put their livelihood at heightened risk in this manner.
Given the avoidable harm done to the trucking industry and the shipping public by prematurely advancing the CSA program — whose flawed BASICs and SMS remain the heart of this SFD proposed rule — it is highly advisable and far more prudent to await completion of the FAST Act’s mandated reconstitution and correction of SMS before attempting to craft an SFD rule, especially one that depends so heavily upon SMS.
Get Your Motor Carrier Trucking Authority & Learn How To Start A Trucking Company today!