Attachment 1 - EFFECTIVENESS CHECKS

    A. Determining the Total Number of Effectiveness Checks to Conduct

        1. After the recall officer (RO) has removed duplicate consignee entries from the master consignee list (MCL) and has determined the total number of consignees, the RO will determine the appropriate table in this document to assign a total number of effectiveness checks in APM that will be performed by on-site verification and by telephone. If there is sufficient information, the RO may decide to group effectiveness checks by special consignee categories (e.g., schools, day care centers, hospital cafeterias, or retirement homes). If the RO decides to group effectiveness checks by special categories, the RO is to apply the appropriate table in this document to consignee category in APM, as provided in Section B of this attachment. The RO and FSIS personnel are to use the timeframes in Table 1 as a goal for completing a substantial portion of verification activities. Verification has begun when FSIS contacts any consignee of the recalling f irm.

        2. ROs are to be aware that large corporate chains which have numerous retail locations may provide a single report for all their locations or individual reports for selected locations, provided the chain has a robust system that allows for reporting recall notification and product disposition. The RO may review such reports to verify the FSIS selected retail locations in lieu of conducting phone checks.

        3. Table 2 is used to determine the number of checks for all Class I recalls when there has been an illness, outbreak, or school distribution (see Section B: Schools and Other Special Consignee Categories).

Table 1. Recommended timeframes for initiating and reporting verification activities within FSIS

* Working days: Working days may include Saturday and Sunday, depending upon the risk associated with a recalled product.

Table 2. Effectiveness checks to conduct and critical limits for all Class I recalls involving an injury, illness outbreak, or distribution to schools.

        4. Table 3 is used to determine the number of checks for Class I recalls when there are no illnesses, outbreaks, or school distribution.

Table 3. Effectiveness checks to conduct and critical limits for Class I recalls when there are no illnesses, outbreaks, or distributions to schools.

        5. Table 4 is used for Class II recalls.

Table 4. Effectiveness checks to conduct and critical limits for Class II recalls.

        6. Table 5 is used for Class III recalls.

Table 5. Effectiveness checks to conduct and critical limits for Class III recalls.*

*Effectiveness checks for Class III recalls will be performed by telephone, unless the RO determines that on-site verification is necessary.

    B. Schools and other Special Consignee Categories

If information is available, the RO may group effectiveness checks by identified special categories (e.g., schools, day care centers, hospital cafeterias, or retirement homes), to mitigate risk to populations that may be more susceptible to foodborne illness. If the RO decides to separate groups by special categories, then each group of consignees should be considered separately. Apply the appropriate table in APM for the recall classification and type of special consignee category to select the number of effectiveness checks to be conducted for each special group. This will have the effect of increasing the number of effectiveness checks to be conducted at these facilities. Schools may also be grouped into a special category of consignees for conducting effectiveness checks during Class II and Class III recalls. During Class III recalls, all checks may be conducted by telephone.

FSIS typically does not conduct effectiveness checks on schools that participate in the School Lunch Program or other assistance program administered by USDA's Food and Nutrition Service (FNS) and that receive reimbursement for the cost of the recalled product by FNS. However, FSIS may determine that effectiveness checks or other actions are necessary at such schools, on a case-by-case basis.

In special limited circumstances, to protect public health, FSIS may decide to conduct a greater number of effectiveness checks than the number provided in the tables. For example, FSIS may increase the number of effectiveness checks if the recall involves a product that has been implicated in human illnesses and the Agency continues to receive reports of new illnesses after the issuance of the Recall Release.

    C. Selecting the Effectiveness Checks

The RO defines the criteria for selecting effectiveness checks in the APM Recall case. The RO should consult with RMTAD for any questions regarding which table in this document is appropriate for effectiveness checks selection.

    1. Using the number of consignees and any decision to group effectiveness checks into special categories, the RO should determine the appropriate table to assign a selection rate.

        a. For a Class I recall with no illnesses, outbreaks, or school lunch distribution, the appropriate table is Table 3.

        b. If the RO decides to group effectiveness checks into special categories (e.g., schools, day care centers, hospital cafeterias, or retirement homes), then each group of consignees is considered separately. Use the tables to determine the number of effectiveness checks to be conducted for each group.

        c. For a Class I recall with school lunch distrbution and retail/restaurant distribution, the appropriate table for the Schools consignee group is Table 2 and the appropriate table for the retail/restaurant consignee group is Table 3.

        d. Grouping consignees into separate categories should always result in an increase in the number of effectiveness checks to be conducted.

    2. Once the MCL has been compiled, the RO, or designee, is to upload the MCL to the APM Recall case, enter the criteria for selecting checks, and follow the steps in APM for selecting checks. APM will generate a list of randomly selected effectiveness checks based on the tables and any grouping selected by the RO.

    NOTE: The RO is to refer to Attachment 2 for instructions on how to select recall effectiveness checks in the event of a widespread system outage or other extenuating situation requiring a contingency plan.

APM will send a daily alert including information on the consignees selected for effectiveness checks to the Supervisory users that will assign the checks. If the recalled product was distributed to consignees in more than one District or Region, the appropriate AO is to assign FSIS personnel to perform the effectiveness checks in their District or Region.

The information that the RO provides to the FSIS personnel conducting the effectiveness checks should include the recall case number, the consignees selected for effectiveness checks, the recommended timeframes for completion, and any other details that may help conduct the verification activities more effectively.

If FSIS personnel are unable to perform an effectiveness check (e.g., a consignee selected for an effectiveness check did not receive the recalled product or is no longer in business) and determine that the check is ineligible, they are to contact the RO through their supervisory chain as soon as possible so that a replacement effectiveness check can be selected and assigned. The RO will use APM to select a replacement effectiveness check by substituting a consignee by random selection or a biased replacement consignee that the RO believes is likely to have received the recalled product. After making this substitution, APM will designate this effectiveness check to the applicable AO for assignment

If the consignee selected for substitution is also ineligible for an effectiveness check, the RO is to select another substitute consignee. The second substitution should be a biased replacement consignee that the RO believes is likely to have received the recalled product. The RO should make a reasonable attempt to find a substitute consignee so the effectiveness check can be completed.

If FSIS personnel are having difficulty locating a substitute consignee that received the recalled product, the RO is to contact RMTAD.

NOTE: There can be no substitutions if all consignees are selected for effectiveness checks.

    D. "Findings of Product in Commerce" is defined as those occurrences where recalled product remains available to consumers.

        1. When the DDM/RD or designee is notified by personnel in their district/region of findings of recalled product in commerce, they will immediately notify the RO.

        2. The RO is to determine whether the findings follow a pattern or trend. During the evaluation, it is important to distinguish between isolated reasons (e.g., the product was removed from the store shelf but was re-shelved by mistake) and widespread systemic reasons (e.g., breakdown in the notification of consignees or delay caused by the schedule of sales personnel). This is important to do, even if the recall itself is effective, because there may be subgroups of consignees that have recalled product that is available to consumers. When a trend is identified, the RO may assign additional effectiveness checks by biased selection to verify that recalled product is not available to consumers. As deemed appropriate and necessary, the RO will notify the Director of the Compliance and Investigations Division (CID), OIEA.

    E. Special Circumstances -- Determining the Need to Consult a Statistician

There may be instances in which FSIS personnel may need statistical guidance when performing recall effectiveness checks. For example, FSIS personnel may not be able to contact consignees selected as effectiveness checks because the consignees are mobile (e.g., the product was distributed to a cruise ship).

In these circumstances, FSIS personnel are to inform RMTAD. RMTAD will contact the Office of Planning, Analysis, and Risk Management (OPARM) statistician assigned to assist with recalls and will refer the statistician to the appropriate RO. The OPARM statistician will work directly with the RO to provide any needed statistical guidance.