Summary Product testing is an inefficient tool for enhancing the safety of fresh fruits and vegetables. The probability of finding a contaminant is very low at the bacterial population densities likely to be found. Product testing is wasteful of resources because of the time and expense of the testing itself, but also because all tested products must be put on hold to await the test results in order to avoid a costly recall should a test return a positive result. Holding tested products requires additional cold-room space, additional handling and additional energy, adding significantly to the cost of the testing program. The resources devoted to product testing are thus not available for more efficient programs aimed at preventing contamination in the field and in the handling facility. For these reasons, product testing is generally not recommended by food microbiologists with expertise in produce microbiology. Micro Testing for Risk Reduction Many product testing programs are based on the assumption that the testing will reduce the risk of contaminated product reaching the marketplace. The counter-argument is that you “can’t test safety into the product.” Both views have some validity. Contaminated product that is removed from the commercial stream does reduce the risk that the contaminated lot will cause illness. But, like risk itself, the issue is not whether, but how much, and at what cost. With practical commercial testing schemes, those that take fewer than hundreds of samples per lot, there is a probability that many, if not most, contaminated lots will not be detected by the sampling scheme. Therefore, testing is a blunt tool as a risk-reduction intervention. Before embarking on an expensive testing program, it is fair to ask if the necessary resources might more effectively be used elsewhere. A detailed risk assessment and preventive interventions based on that assessment might be a more efficient use of resources than trying to remove contaminated products farther up the handling chain. End-product testing is antithetical to a HACCP (Hazard Analysis Critical Control Point) approach to risk reduction. HACCP requires preventive interventions at critical points in a process where the risk of contamination is high. When HACCP was introduced to the space program many years ago, it was designed to replace end-product testing for contamination because testing was seen to be relatively ineffective as a risk-reduction intervention. HACCP has now been widely accepted by food regulatory authorities throughout the world as a replacement for product testing. Probability of Finding a Contaminant Pathogenic bacteria are rarely found on fresh-cut vegetables. Foodborne illness outbreaks are rare events. Consequently, the results of tests for pathogens would be expected to nearly always be negative. But what is the probability of finding a pathogen if it is present? The authoritative publication, “Microorganisms in Foods 7 – Microbiological Criteria for Foods Book 7,” points out that if one were to take 60 samples from a lot that was known to be contaminated at the 0.5-percent level, one would have a 74-percent probability of accepting this contaminated lot. That is only a 26-percent probability of actually detecting the pathogen. If the pathogen were present on only 0.1 percent of the product, those 60 samples would have a 94-percent probability of missing the pathogen. Evidence suggests that when human pathogens are present on fresh produce, it is usually at a very low frequency. The probability of detecting a human pathogen on contaminated produce is probably very, very low. Product testing is likely to be effective in those rare cases where contamination is so high that the test has a high probability of finding it. Situations where contaminated water is used to spray plant-protection chemicals or where contaminated nutrient solutions are sprayed on a crop can result in a very high incidence of contamination. But detecting these contamination events through testing is a most inefficient approach. It would be far better to monitor water quality and the microbila quality of nutrient solutions than test the fruits or vegetables after application. So once again, pathogen testing is a blunt tool, even in these situations. How do You Sample for It? Microorganisms on produce, or in the environment, are typically not uniformly distributed. They may be found in some parts of a field but not in others, or on some individual fruit or vegetable and not on others. This makes sampling and testing a difficult, and often complicated, enterprise. The probability of finding what you are looking for, if it is present at all, depends largely on the ability to test enough samples to find it. In the case of human pathogens on raw products in the field, or on finished products in a packing shed or processing facility, they are usually not present at all. If they are present, they may be unevenly distributed and present in low numbers. This makes them difficult to find, even using the most sensitive of tests. If Salmonella occurs in one bag of lettuce out of every thousand, it would take a lot of test samples to have a reasonable chance of detecting it. This is the central problem of pathogen testing. While there are very fast, very sensitive test methods that can detect pathogens, the number of samples required may be so great that it is impractical to do the tests. Data from USDA and FDA domestic and imported produce sampling surveys indicate that human pathogens are found on fresh produce infrequently and in low numbers. However, recent outbreaks associated with produce have prompted buyers to demand product testing as a condition of sale. This reactive strategy is not an effective means of assuring consumer safety. Where to Test? Because of the statistical limitations of testing for pathogens present, if at all, at low numbers, if one wants to detect pathogens through testing, it is rational to use available knowledge of the likely location and pattern of contamination, should such knowledge be available. In the case of end-product testing in a processing facility, knowledge of where in the field a bin of lettuce came from is usually not available. In addition, knowledge of the potential sources of contamination associated with a bin of lettuce is usually not available. Consequently, product testing is constrained by assuming a uniform distribution of a putative pathogen, though this assumption is almost certainly false. The result is that the sensitivity of product testing is dramatically lower than the sensitivity of a test would be where environmental knowledge informs the test. For this reason, nearly all processors who perform product testing take their samples in the field prior to shipment to the processing facility. In this way, sampling can be concentrated in the parts of the field that may be nearer riparian vegetation or nearer wild lands or grazing animals. Such knowledge of likely sources of contamination can increase the statistical power of the testing regime. In addition, field testing can detect a contaminant before the products have been processed and possibly co-mingled with raw material from other lots or other fields. This would greatly simplify investigation and ensuing recall should a contaminant be detected. For these reasons, product testing in the processing or handling facility is the least-efficient way to test for the presence of pathogens. And, as discussed above, testing for pathogens is already a rather inefficient tool. How Long Does it Take to Get Results? Because of the perishability of fresh fruits and vegetables, it is imperative that test results be returned rapidly so that appropriate action can be taken. In the case of pathogen testing, this really means that the lots being tested must be held until the test results are returned. This obviates the enormous cost of a recall should there be a positive pathogen test result. Holding all of the products being tested requires much additional storage space, additional handling and additional energy costs to keep the products cold. The additional holding time necessarily reduces the shelf life and quality of the products being held. The Value of Pathogen-Testing Programs The USDA MDP pathogen-testing program, FDA testing programs and individual companies’ testing programs have been in place for some time, though the MDP program has been terminated. The results of large testing programs, whether public or private, indicate that pathogen contamination of fresh fruits and vegetables is the exception. Hundreds of thousands of field tests of leafy greens by private companies suggest contamination rates of roughly one in 10,000. Contamination rates of some other commodities, according to USDA and FDA results, may be higher, but still in the range of one in 1,000. At these contamination rates, testing 60 samples, the standard sample size in commercial produce testing, is very unlikely to detect a contaminant. So what is the value of these testing programs? Large testing programs can inform us of general rates of contamination in a population of products. Large testing programs may uncover regional differences in contamination rates. Properly constructed, large testing programs can help validate whether specific interventions, such as water purification or compost preparation, are reducing contamination rates. What pathogen testing programs are not particularly good at is detecting and removing contaminated lots of produce from commerce. Testing is expensive and intrusive and unlikely to detect pathogens, even in the exceptional case when they are present. Commercial operations simply cannot test enough samples, or large enough samples, to have a reasonable probability of detecting pathogens. Conclusions Product testing for the presence of human pathogens is an insensitive tool and an inefficient use of food safety resources. Those resources are likely to be more effective in enhancing food safety if they are invested in HACCP-based preventive programs. If testing for pathogens is to be done, it is more effective to perform the testing in the field before the products reach the handling facility.

  • Keith Warriner

    I agree that test-and-release programs provide as much protection as a paper hat there is virtue as a means of verifying the effectiveness of food safety systems. With respect to fresh produce, it the inputs (water, soil) is good then the product should be fine. Still post-harvest interventions are key and we have lacked behind here in believing washing actually does the job.

  • Paul_LK

    Very good points. The USDA MDP pathogen-testing program has been repeatedly discussed on this site over the last few years. The value of the testing program must be considered in the context of the probability of finding a positive result. Putting resources into testing water quality and other sources of potential contamination (rather than end product) makes good sense and prioritizes resources for “safety.” This is where a good HACCP program is needed. Unfortunately, most of the public (and others unfamiliar with sampling) get a false sense of security from knowing that testing is being done on their foods.

    It would be interesting to examine the statistics associated with the sampling results of the Salmonella peanut contamination from a few years ago, and then comparing these with the investigation inspection results at the implicated plant. A good case study, I suspect.

  • Android

    Excellent article. Appreciate your thoughts Devon

  • Kathryn

    Well, that’s a grim view on produce testing. I see the point that sampling for human pathogens in fresh produce may not have cost-benefit or efficacy on it’s side at this moment in time, but there is certainly reason to be hopeful for such processes to become more pragmatic in the future. Some of the biggest research institutions focus on developing more effective, rapid, and specific testing that also can yield a positive result for smaller microbial populations. Science moves forward. And just because HACCP was implemented to shift focus to the critical control points throughout production, does not mean that end of the line testing is at no value. Original intent is not a reason to avoid adaptation in food safety programs. I agree that a solid HACCP plan can be hugely effective in preventing contamination during processing, but no process is 100% effective all the time. Not to mention, development and maintenance of such programs requires constant attention. End of the line testing is often an instrumental tool in weeding out holes, or issues in the HACCP program. I feel there is a lot of work to be done to improve the safety of our produce, even if instances of contamination are statistically rare.

  • Karl

    I admire and respect the
    work of Dr. Zagory. He has contributed significantly to the produce
    industry. He is a leader we need to listen to, and learn from his
    experience, This article is exactly right when discussing traditional,
    conventional testing; to sum his well written article, to test a sample is to
    test the sample, in other words to draw any conclusions about the load from
    which the sample was drawn is ludicrous. I would however, like to
    offer some insights on a number of validated methods now in the industry which
    are being taken very seriously by government regulators and provide value to a
    few of Devon’s comments. (As a disclaimer, I am a partner in a nationwide
    laboratory that has pioneered validated PCR programs.) 1) Traditional testing
    methods are archaic, 2) No test (ever) should replace a valid and robust food
    safety program, 3) Validated, Direct-to-PCR programs using modern sampling
    methods are producing confidence targets approaching and often exceeding 95% of
    all finished product, 4) Results using these programs are achieved in less than
    30 minutes thereby reducing or eliminating the need for buffer storage, 5)
    Validated studies over the last 4 years demonstrate 1 out of 800 samples using
    this technology are suspect for contamination, 6) The cost of this technology
    is exceptionally low. I absolutely do not wish to take anything away from
    Dr. Zagory; he is a consummate professional and I am excited he has highlighted
    a number of the problems with traditional testing; my comments are intended to
    only shed light on new and emerging technology that has overcome many of the
    testing obstacles now in the industry.

  • Robert

    Right Kathryn. Perhaps the problem is the test method. Our electronic imaging test has detected internalized E. coli in 3 seconds, negatives in 2 seconds. That can be fast enough for 100% conveyor sampling at the Processors’ inspection station.