Methods Used to Score Products on Health

Overview

To rate a product's health performance, GoodGuide's system takes into account the potential impacts of using a specific product on your health.  We assess health impact by combining 1) negative metrics (such as whether the product contains toxic ingredients of concern or ingredients that have low nutritional value) with 2) any available positive metrics (such as whether the product has been certified as safe or healthy by a credible third-party).  For personal care and household chemical products, scores are based on the presence of ingredients of concern.   For food products, scores are based on the nutritional value of ingredients. For toy products, scores are based on the presence of chemicals or metals of concern.

Levels of Health Concern Defined

To identify ingredients of concern, we employ the science of health hazard assessment and rely on lists of chemicals that have been labeled hazardous by various authoritative organizations.  GoodGuide tracks whether chemicals are recognized or suspected of causing any of twelve major types of human health problems, ranging from cancer to endocrine toxicity to skin or eye toxicity.  We combine this hazard data with information on how potent a chemical is, whether it is frequently detected in humans, and whether it has been adequately studied in toxicity tests in order to assign ingredients to four levels of health concern: none, low, medium and high.

 An ingredient raises no health concern if

• It is not on any of GoodGuide’s lists of toxic chemicals which cause suspected or recognized health effects,
• It has not been detected in human tissue or urine
• It is not a high production volume chemical that lacks safety data

 An ingredient raises a low level of health concern if

• It exhibits two or fewer suspected health effects, and/or
• It has a relatively low toxic potency for inhalation and ingestion exposures, and/or
• It is only occasionally detected in human tissue or urine, and/or
• It lacks at least half of the six basic toxicity tests required to assess chemical safety

An ingredient raises a medium level of health concern if

• It exhibits three or more suspected health effects, and/or
• It has a relatively moderate toxic potency for inhalation and ingestion exposures, and/or
• It is regularly detected in human tissue or urine

 An ingredient raises a high level of health concern if

• It exhibits one or more recognized health effects, and/or
• It has a relatively high toxic potency for inhalation and ingestion exposures, and/or
• It is frequently detected in human tissue or urine

 The Difference Between Recognized and Suspected Health Effects

Chemicals are identified as recognized toxicants based on the hazard identification efforts of authoritative national and international scientific and regulatory agencies. To date, such efforts have been focused on only a few types of toxicity: GoodGuide utilizes lists developed under California's Proposition 65  to identify recognized carcinogens, reproductive toxicants, and developmental toxicants. We use a peer-reviewed article in the medical journal Lancet to identify recognized neurotoxicants. Chemicals are listed on Proposition 65 after scientific peer review and regulatory rulemaking, which incorporates the hazard identification efforts of a variety of other authoritative bodies like the International Agency for Research on Cancer and the National Toxicology Program. A substantial weight of toxicological or epidemiological evidence supports the decision to list a chemical as a recognized health hazard under Proposition 65. Stakeholders that believe a chemical does not cause a recognized health effect have the opportunity to argue that the evidence does not support identifying the chemical as a hazard. If a chemical is listed under Proposition 65, such arguments failed to convince neutral scientific and regulatory experts.

Chemicals are identified as suspected toxicants based on reports in the scientific or regulatory literature, or on information abstracted from major toxicological databases. Lists of suspected toxicants are available for twelve health effects. Suspected toxicants possess evidence that they can cause specific adverse health effects, but no authoritative hazard identification is currently conducted by regulatory agencies or scientific organizations for that health effect. Inclusion of a chemical on a "suspected" list should be viewed as a preliminary indication that the chemical may cause this effect, rather than a definitive finding that it does. To identify suspected toxicants, information was abstracted from the principal toxicology text books (such as Casarett and Doull's Toxicology), medical journal articles, regulatory actions, and international chemical hazard resources (such as the European Union). The weight of toxicological or epidemiological evidence supporting suspect hazard identification can vary significantly between chemicals. For example, evidence from two different laboratory species indicates that acetonitrile can cause cardiovascular toxicity. In contrast, there is overwhelming evidence that carbon monoxide causes cardiovascular toxicity in humans. On this website, these differing amounts of evidence both lead to designation as a "suspected" toxicant, because no agency authoritatively compiles lists of cardiovascular toxicants. Identifications made by regulatory agencies or scientific references have often undergone peer review, but no administrative process has occurred that allows debate over the toxicity of a chemical to be resolved conclusively.

A Note about Controversial Ingredients 

GoodGuide separates ingredients into two categories - "ingredients of concern" and "controversial ingredients," and only the first category contributes to a product's rating.  To be labeled an ingredient of concern, the chemical at issue needs to have been identified as a potential health hazard by a source that GoodGuide judges authoritative (e.g., a regulatory agency like EPA or a scientific group like the National Toxicology Program). We do this to make sure our ratings are based on the best available scientific evidence.  In some cases, a chemical has not been identified as a hazard by authoritative sources, although it is the subject of current debate as to whether it might have adverse effects. Goodguide labels such chemicals as "controversial" ingredients which a consumer might be interested in avoiding, but because the evidence of hazard is inconclusive, this does not contribute to the product score.   

Scoring Personal Care and Household Chemical Products

To assign a health hazard score to a personal care or household chemical product, GoodGuide counts the number of its ingredients that are categorized as low, medium or high concern. Products that contains one or more ingredients that raise a high level of health concern are scored 0 - 1; products that contains one or more ingredients that raise a medium level of health concern are scored 2 – 4; products that contains four or more ingredients that raise a low level of health concern are scored 5 – 6;  products that contains one to three ingredients that raise a low level of health concern are scored 7-8; and products that contain no ingredients of concern are scored 10.

Adjusting Scores to Account for Data Gaps

GoodGuide adjusts a product's health score if information is missing that is required to evaluate its potential impact.  A comprehensive health evaluation requires 1) a complete list of a product's ingredients (with sufficient detail about chemical identity to allow ingredients to be checked against hazard lists) and 2) information about the percent composition of a product (to characterize potential exposures and evaluate compliance with regulatory restrictions). Unfortunately, manufacturers are generally not required to provide either of these types of information for most consumer products.     

1) Adjustment for products that lack full ingredient data. For household chemical products, there is no current regulatory requirement that companies disclose full ingredient lists.  For personal care products, ingredient disclosure is required, although it may include generic ingredient names that are not specific enough to support hazard evaluation.  To create an incentive for full ingredient disclosure, GoodGuide caps a product’s score if it lacks complete ingredient data or lists generic names that do not support chemical-specific evaluations.  The caps applied are described in the following table:

Rating Cap	Amount of Ingredient Information Available
0	No information available
4	Some ingredient information available, but list includes generic categories (e.g., surfactants) that are not sufficiently specific to identify actual ingredients
8	All ingredients are disclosed with the exception of fragrance, which is listed as a generic category
10	All ingredients are disclosed including fragrance constituents (or the company warrants that no chemicals of concern are included in its fragrance)

 2) Adjustment for products that lack percent composition data. For household chemical products and most personal care products (with the exception of sunscreens), there is no current regulatory requirement that companies disclose percent composition data.  This precludes evaluating whether a product is a potentially significant source of exposure to a chemical (e.g., presence of an ingredient at less than 0.1% of a formulation is unlikely to pose a risk, while presence of that ingredient at 10% of a formulation could be a significant source of human exposure).  The absence of percent composition data also complicates the evaluation of whether a product is in compliance with applicable regulatory restrictions.  Both regulatory agencies and trade associations manage the potential health risks of products by defining thresholds below which an ingredient is deemed safe as used (e.g., use of ingredient x is acceptable provided that it does not exceed 5% of a product's formulation).  Unless a product label discloses the percent composition data required to evaluate compliance, GoodGuide caps a product's score at 8 if it contains ingredients that have been restricted by regulatory agencies or trade associations.

"Below Threshold" Adjustments

GoodGuide adjusts a product's health score upward if information indicates that an ingredient does not pose a health or regulatory concern as it is used in a product.  This adjustment has the effect of suppressing the contribution of that ingredient to the product's health score.  Such adjustments are indicated in the "Product Ingredient List" section of GoodGuide product pages with a "Below Threshold" icon and an explanation of the basis for suppressing an ingredient.   Scores may be adjusted for the following reasons:

• An authoritative third-party (such as EPA's Design for the Environment program) has reviewed a product and approved its formulation.
• An authoritative third-party (such as a regulatory agency) has reviewed the ingredient and determined that its use in a certain type of product (e.g., rinse-off hair products) does not result in significant exposure or health risk.
  
• Percent composition data indicate the level of the ingredient is below applicable regulatory or trade association threshold levels that define safe use.
  
• Percent composition data indicate the ingredient is present at a concentration below a de minims level of 0.1%.
  

Scoring Food Products

GoodGuide’s food health and safety ratings are calculated based on three main attributes: nutritional value, presence of potentially hazardous food additives, and production practices. Nutritional value is at the heart of the rating, and is based on an accepted method of nutrient assessment called the “Ratio of Recommended to Restricted nutrients” (RRR). The RRR calculates the ratio of “good” to “bad” nutrients, essentially providing an overall picture of a food’s nutrition profile. The list of recommended nutrients includes protein, calcium, iron, vitamin A, vitamin C, and fiber. The list of restricted nutrients includes calories, saturated fat, cholesterol, sugar, and sodium.* Each food is then further analyzed to determine whether certain nutrients of public health significance (sugar, sodium, cholesterol, saturated fat, and trans fat) are present in levels that exceed recommended thresholds. An added level of assessment is conducted on cold cereals to determine the quality of carbohydrates (specifically fiber and sugar) in these products. The thresholds we use come from a variety of authoritative sources, including the Institute of Medicine (the scientific body that provides evidence for the Dietary Guidelines for Americans) and the World Health Organization, and are applicable for a 2,000 calorie per day diet.

There is mounting evidence that the benefits of any particular food go beyond just nutrient content. As a result, GoodGuide’s food health and safety ratings also take into account the presence of certain additives and the way a food is produced - health issues outside the traditional focus of nutrient content. Positive adjustments are issued to foods that are certified organic, made without genetically modified organisms (GMO-free), and produced without the use of recombinant Bovine Growth Hormone (rBGH, dairy products only). Negative adjustments are issued based on detection of pesticide residues (fresh produce only) and the presence of added sugars or salt (baby and toddler foods only). Finally, a rating may be capped by the presence of a potentially hazardous food colorant or markers of high-processing. The data used to quantify these issues comes from several third-party data sources, including the UK Food Standards Agency, the United States Department of Agriculture, and The Center for Food Safety. Data quality and accuracy is of the utmost importance to us, and we are consistently looking for ways to refine and improve our food rating methodology.

*For fresh fruits and vegetables, the recommended nutrients include fiber, protein, vitamin A, vitamin C, calcium and iron and the restricted nutrients include calories, saturated fat, cholesterol and sodium. For baby and toddler foods, the recommended nutrients include protein, vitamin A, vitamin C, calcium, and iron and the restricted nutrients include sodium and sugars.

Scoring Toy Products

Toys are scored based on the potential that children could be exposed to toxic metals or chemicals as a result of their material composition.  GoodGuide acquires and scores toy testing data from various third-party sources (and sometimes conducts product testing in-house). Test results for each toy and tested contaminant (typically expressed as parts per million, or ppm) are scored using GoodGuide's 0-10 scale. Toys with no detectable concentration of a particular contaminant rate 10; toys with trace concentrations of a contamination rate 8; toys with moderate concentrations rate 5; toys with high concentrations rate 2, and toys that exceed  Consumer Product Safety Commission limits on total metal concentrations rate 0.  The ppm thresholds used to define trace/moderate /high concentrations vary by contaminant and parallel those in use by our other toy testing partners. In the case of chlorine and bromine, detected concentrations were used as an indicator of the potential presence of PVC and polybrominated compounds.

It is important to note that test results are typically for total metal, not soluble metal concentrations.  Soluble metal testing requires more expensive and destructive tests, but is the preferred methodology for most existing Consumer Product Safety Commission standards for toys, because the test conditions are thought to more accurately reflect the amount of metal a child mouthing a product might be exposed to.  As a result, GoodGuide's test results cannot be used to assess a toy's compliance with a CPSC standard (except in the case, of lead, where the standard applies to total metal concentrations).