GoodGuide’s Methodology

Overview

GoodGuide aggregates and analyzes data on both product and company performance. The team employs a range of scientific methods—health hazard assessment, environmental impact assessment, and social impact assessment—to identify major impacts to human health, the environment, and society. Each of these categories is then further analyzed within specific issue areas, such as climate change policies, labor concerns, and product toxicity. Currently, GoodGuide’s database includes over 1,100 base criteria through which we evaluate products and companies.

As an example, for health performance, GoodGuide’s system takes into account both the impacts of a company’s operations on its workers and local communities, and the impacts of using the product on an individual’s health. Our team has gathered data on important health hazards such as cancer risks, reproductive health hazards, mutagenicity, endocrine disruption, respiratory hazards, and skin and eye irritation. The science team currently uses a health hazard assessment process that allows us to rate thousands of products along a standard set of criteria. It should be noted that while these ratings are not “risk assessments” of products or chemicals, they do highlight potential hazards associated with the use of the products.

For environmental performance, GoodGuide aggregates data on the life-cycle impacts of products, from manufacturing to transportation to use to final disposal. For companies, environmental impact categories include environmental emissions (and their impacts on air, water, land, and climate), natural resource impacts, and environmental management programs. GoodGuide uses these categories to generate overall environmental performance ratings for companies.

For social issues, GoodGuide aggregates data on the impacts companies have on their employees (such as compensation, labor and human rights practices, diversity policies, and working conditions), consumers, communities, and even animals. The social scoring system also brings together information on corporate governance, disclosure policies, and overall practices.

All the data used by GoodGuide are assessed for their credibility, quality, importance, and usability to determine which datasets are suitable for use in our ratings. A more refined assessment is then performed on each of the criteria, including a statistical analysis of the data and a benchmarking analysis to related criteria.

Methodology

Different types of information flow into GoodGuide’s system: absolute measures, relative measures, and binary measures. Absolute measures describe measurable activities of a company or product. These might include the pounds of toxic air emissions released per year, the CEO’s salary, or the amount of money a company donated to charity. Relative measures are scores, such as a numerical grade of &ldquot;6.5 out of 10&rdquot; or a textual grade of 'bad' to 'excellent'. Binary measures (or Yes/No measures) indicate whether a product or company does or does not have specific characteristics or practices. For example, a product may or may not have earned an environmental certification, or a company may or may not have a policy on animal testing.

These measures are then used to create GoodGuide’s ratings. To calculate a single rating for a product or company, we convert all of the existing measures into a 0 to 10 score, where a score of 10 is the best and a score of 0 is the worst. Given the different aspects of the measures used in GoodGuide’s system, this conversion is done using a method called standardization. This is the same method used to grade students “on a curve” as in an American school’s grading system. What this method does is to make each variable comparable with all the other variables in the system. For example, if 9 out of 10 companies have implemented one particular policy, then the value of having implemented that policy is likely smaller than the value of a policy that only 2 out of 10 companies have implemented. Standardization makes the measures for these two policies comparable. Finally, GoodGuide’s ratings system includes a spread factor and a statistical technique called Windsorization that allows us to align the standardized scores across our 0 to 10 scale, and helps to present differences between products and companies.

GoodGuide rates two types of companies: (1) large, publicly traded companies that are legally required to disclose information on their programs, policies and performance on environmental and social matters; and (2) small, usually private, companies for which these legal requirements don’t apply. GoodGuide recognizes nevertheless the value of many of these smaller companies as market leaders in some segments and understands they are relevant for our users. As such we research these companies based on company self-reported information and third party sources including the FDA, USDA, Department of Justice, EPA, DOL, union news, and other sources. The ratings of these smaller companies are then aligned with the ratings of the larger companies based on the specific characteristics of the companies and the nature of the issue being rated. For example, a small company can have lower environmental impact than a large company if it uses a smaller volume of toxic chemicals or has local supply chains. This may allow smaller companies to reach higher ratings in some issues. At the same time, small companies should be held to the same standards as large companies in matters of workers rights, diversity, etc.

GoodGuide recognizes that even the most quantitative assessment of environmental, health, or social issues requires value judgments about the relative importance of various issues. For example, rational people can disagree over the relative importance of worker treatment in evaluating a product or company. We have used our best scientific judgment in building our current ratings, and on our website we offer a feature to let users “filter” products and companies based on their personal values and preferences. In the near future, we will allow visitors to create personalized ratings based on their own concerns and weightings.

GoodGuide’s science team has built a system for weighting the variables used in the ratings. These weights are determined by teams of experts in each field and look at both (A) the relevance and importance of each criteria (e.g. cancer versus skin irritants) for a specific product or industry, and (B) the quality of the data and their provider. The quality and credibility of the data provider is assessed based on the track record of the provider and by evaluation from experts in each field. The quality of the data itself is assessed by validation against other sources and by a detailed assessment of the data collection process used by the data provider.

In some cases, data is unavailable for a specific attribute or impact of a product or company. This may be because the team has not yet identified a credible data source for a given issue or topic. It may also be that the data is not publicly available because companies have not disclosed critical information. GoodGuide is working collaboratively with key stakeholders around the world, including government agencies, non-profit organizations, private research agencies, and companies to promote the quantity and quality of disclosure of important data to the public.

Ratings

Health Ratings

The health criteria in GoodGuide were created with the intention of being hazard-based in order to be precautionary in the screening of potential hazards of product ingredients. The initial screening of potentially hazardous ingredients is list-based. Criteria and data sources for lists were selected based on an emerging consensus from progressive municipal governments’ Environmentally Preferable Purchasing (EPP) policies, governmental sources (e.g. the National Institutes of Health), recognized NGOs (e.g. Clean Production Action), third-party standards development entities (including NSF), and others.

Weighting of Criteria and Data Sources

There is an inherent challenge when weighting independent health criteria against each other for the general public, which contains multiple sub-populations with different health sensitivities and concerns. The GoodGuide team developed an internal ratings system for the Health criteria based on weightings provided by our science team. These weights consider both the hazard level of each ingredient and the quality of the data available for ingredients with specific potential health hazards.

Data Gaps/Emerging Concerns:

The universe of products and hazards that GoodGuide covers is very large, and in a number of cases there is no existing regulatory requirement for disclosure of critical data for products. As a result, there are sizable data gaps in the public information available for GoodGuide’s ratings. These data gaps include both gaps of information about the hazard level of particular ingredients and gaps of information on the product formulation.

GoodGuide’s team of scientists works very hard to cover data gaps on the hazard that a particular ingredient may pose for our users. We partner with leading governmental sources, NGOs and research institutions, to be at the forefront of knowledge about the product categories that we cover. Currently, GoodGuide tags ingredients for which there is insufficient data on their level of safety as “Controversial Ingredients” and sets ratings thresholds for products containing these ingredients. These are ingredients that warrant some concern but that have not been fully proven as hazardous. GoodGuide is continuing to add to this list as new information becomes available or more concerns and data emerge.

GoodGuide is also continuously working with manufacturers and other data providers to cover data gaps related to product formulations. This issue is particularly relevant for Household Cleaning products because manufacturers are not required in the US to disclose most of the ingredients used.

Ingredient Information for Household Cleaning Ratings

One of GoodGuide’s goals is to provide accurate and fair ratings on all the categories we cover. The Household Cleaning category is particularly challenging because manufacturers often do not disclose ingredient information. With the health and safety of our users in mind, GoodGuide chose to cap the ratings of these products if there isn’t sufficient ingredient information to assess the hazard level of a product. GoodGuide’s ratings for this category of products are limited by the following thresholds:

GoodGuide is open to working with manufacturers to update information on their products. We are currently working on processes that will allow manufacturers to disclose critical information (e.g. details on fragrance) while protecting trade secrets. GoodGuide is also continuing to partner with advocacy groups, third party standards organizations and others involved in chemicals policy reform to track the issue of disclosure, particularly around Fragrance formulations and other information protected under current chemical regulation as Confidential Business Information (CBI).

Health Risk Assessment

GoodGuide does not yet assess exposure risk in its rating of the potential health impacts of a product’s ingredients. Lack of data and the challenge of calculating ratings for each specific type of user, makes this issue a very hard one to tackle. GoodGuide would like to add an exposure assessment (e.g., dermal application vs. ingestion), and consider both the concentration of chemicals in a product formulation as well as how the formulation itself may change the hazard of a specific ingredient.

Food Health Ratings

GoodGuide’s food health ratings are calculated based on a combination of a nutrition rating for each product, existence of potentially hazardous food additives, and product or company certifications. The base nutrition rating for each product is calculated based on a nutrition assessment method called the “Ratio of Recommended to Restricted nutrients” (RRR). This method calculates the ratio of “good” to “bad” nutrients. The list of recommended nutrients includes protein, calcium, iron, vitamin A, vitamin C, and fiber. The list of restricted nutrients includes calories, saturated fat, cholesterol, sugar, and sodium. The RRR score is then adjusted based on the existence of certain nutrients or ingredients above a certain threshold. This is done to highlight healthier products for our users. Currently GoodGuide has set thresholds on sugar, sodium, trans fats, hydrogenated oils, and unnecessary enriched or fortified ingredients. Finally, a product’s nutrition score may be capped by the existence of chemicals of concern such as artificial sweeteners in the product.

Environmental Ratings

GoodGuide’s environmental ratings are composed of both product and company performance assessments.

Company environmental performance is assessed based on three aspects: resource management; environmental impacts; and environmental management. Resource management looks at inputs used by the company in its manufacturing processes. Specifically, we look at materials, energy and water use. Environmental Impact assesses the outputs of the manufacturing processes and how they may impact the environment. This includes Ecosystems and Biodiversity; Air Pollution; Toxic Waste; and Global Warming. Among other measurements, GoodGuide uses Economic Input Output Life Cycle Assessment (EIO LCA) data to assess the overall environmental impact of the manufacture, use, and disposal of a product. This technique provides a comprehensive estimate of impacts of the full supply chain of the industry. Environmental Management looks at issues that relate to the way the leadership of the company manages the relationship between their business and the environment. For example, GoodGuide looks for disclosure practices, auditing processes, and compliance issues. 

Product environmental performance is assessed for a product category in aggregate using process-based Life Cycle Assessment. These data are currently being generated for a select number of product categories for which there is data available. GoodGuide plans to gradually expand the amount of LCA data used to rate products as more and better data becomes available.

Social Ratings

GoodGuide’s Social ratings assess the impact of a company on its major non-financial stakeholders: consumers, employees and the communities in which it operates. Consumer issues covered include company-level product quality and safety criteria, as well as information about customer satisfaction policies. Employee criteria cover policies, programs and outcomes for company employees and other workers in the supply chain related to: working conditions, benefits, workplace diversity, training and development, labor rights, and human rights. Criteria on Community impacts encompass community dialogue policies, participation in community support programs, philanthropy and community-related controversies, including involvement in countries with oppressive regimes. Governance criteria encompass corporate ethics, as well as company financial transparency and reporting on environmental and social issues.

Data Quality Management

GoodGuide is working to be the most reliable source of information on products and companies. We employ quality assurance and quality control (QA/QC) processes to ensure that GoodGuide maintains the highest standards of data integrity and reliability. Our QA/QC protocols include procedures to proactively resolve potential data issues, minimize data errors, and ensure a rapid and effective response to questions about potential errors.

GoodGuide’s data assessment protocol involves two components: affirmative data validation and responsive quality assurance. The affirmative data validation protocol details steps for routine data testing at each data processing stage and is led by the science team with support from our engineering staff. The responsive quality assurance protocol provides procedures to investigate and respond to complaints.

Addendum: Ratings of Independent Companies

GoodGuide conducts independent research on companies that are not covered by our Socially Responsible Investment (SRI) research partners. The companies that GoodGuide researches are either privately held or are too small to be covered by SRI research firms. GoodGuide also covers subsidiaries of larger publicly-held parent companies if there is evidence that the subsidiary has independent management as well as environmental and social policies and practices that are distinct from those of the parent company. This may be the case, for example, when a natural personal care company or an organic food manufacturer is bought by a larger corporation. Subsidiaries for whom such independence has not been determined receive the parent company score.

Sources

GoodGuide researchers search for all publicly available material that is relevant for understanding corporate environmental and social policies and performance, including certifications, awards, fines or penalties, lawsuits, and public controversies. At present, our research covers:

• All materials that are published on the company website, including corporate social responsibility reports, certifications, awards, mission statements, policies, and environmental and social performance data.
• News dating back 3 years from national, international and regional news sources.
• Recalls listed by the FDA and USDA.
• Warning letters posted by the FDA.
• Department of Justice lawsuits.
• EPA listings of environmental-related fines, penalties, partnerships or awards.
• Emissions data from the EPA’s Toxics Release Inventory program.
• DOL listings of labor-related fines, penalties, partnerships or awards. This includes information from departments that cover issues related to worker safety; diversity and equal employment; wages, overtime, and other employment standards; and benefits.
• National Labor Relations Board rulings.
• Ratings by the Better Business Bureau.
• Union news, reports and press releases. Union coverage includes those who represent workers in the industries in which the company operates, as well as the AFL-CIO and Change to Win organizations.
• News, reports and press releases from international labor and human rights organizations.
• News, reports and press releases from environmental organizations.
• News, reports and press releases from organizations that cover the industry in which the company operates.

Criteria

The company research conducted by GoodGuide covers the environmental and social policies, programs and performance of a given company. Companies are considered to be responsible for their supply chains, even if they do not manufacture their products in-house. As a result, companies are evaluated on the policies and performance of their supply chain relationships, including required codes of conduct, certification mandates and supplier audits, as well as on supplier data when available.

At present, companies are rated on the following aspects of corporate policy and performance:

Environmental Criteria

Resource Management

• Materials — Company efforts to reduce resource consumption. This includes policies, programs and performance on materials recycling, reduction and reuse.
• Water — Company efforts to conserve water and reduce water consumption. This includes policies, programs and performance on water minimization and reuse.
• Energy — Company efforts to reduce energy consumption and use sustainable sources of energy. This includes policies, programs and performance on energy reduction; the purchase of alternative energy such as solar, wind and biofuels; alternative energy production for own use; and excess production for the energy grid.

Environmental Impacts

• Water and Land Pollution — Company track performance on water and land emission reduction. Includes data on TRI emissions and EPA penalties and fines.
• Air Pollution — Company performance on air emission reduction. Includes data on TRI emissions and EPA penalties and fines.
• Biodiversity — Company efforts to reduce biodiversity impact. This includes policies, programs and performance on sustainable agriculture and resource extraction, protection of livestock and genetic crop diversity, protection of endangered plants, and prevention of soil erosion.
• Global Warming — Company efforts to reduce climate impacts. This includes policies, programs and performance to reduce CO2 emissions, including fleet reduction programs, as well as the purchase of carbon offsets.

Environmental Management

• Environmental Management — Company development and use of environmental management systems, as well as more general efforts to integrate environmental considerations into business decisions throughout the company.

Social Criteria

Consumer Issues

• Customer Satisfaction — Company efforts to maintain high levels of customer satisfaction. This includes policies, programs and performance related to tracking and responding to customer concerns.
• Quality and Safety — Company policies, programs and performance related to product quality and quality management systems. This includes product safety management certifications, independent QA testing and independent quality assessments such as consumer reports, product quality awards, recalls, product suits, and a focus on non-toxic and organic products.

Social Issues

• Community Engagement — Company engagement with local communities, including input on company impacts. Community engagement includes three components: specific policies on stakeholder dialogue, participation in local initiatives as reflective of commitment to local community, and broad commitment to social responsibility.
• Philanthropy — Company charitable giving, including levels of financial commitment.

Governance Issues

• Ethics — Company ethics policies, programs and performance. This includes the existence and scope of codes of conduct, awards for good governance, and governance controversies such as anti-trust suits.
• Transparency and Accountability — Company accountability and transparency to stakeholders, as well as auditor and board independence. This includes whistleblower protections, truth in labeling policies, policies on accuracy of financial records and ethics for senior financial executives, and separation of executive and advisory board functions.

Worker Issues

• Working Conditions & Benefits — Company efforts to promote good working conditions and quality of life for its employees. This includes policies, programs and performance on employee safety, benefits, remuneration and work-life balance.
• Diversity — Company efforts to promote a diverse workforce. This includes anti-discrimination and diversity policies, programs and performance with respect to race, sex, disability, religion and ethnicity.
• Labor and Human Rights — Company efforts to promote labor rights and human rights for its employees and other workers in its supply chain. This includes policies, programs and performance on core labor rights (such as freedom of association), bargaining in good faith with unions where relevant, and the prohibition of child and forced labor.
• Training — Company efforts to promote training for its employees and internal opportunities for advancement. This includes policies, programs and performance on training and development, internal promotion, cross-functional assignments and educational reimbursement.

Disclosure

Company scores implicitly contain a disclosure component, since companies may have additional internal operations information that is not shared publicly and that is therefore not counted in the ratings. GoodGuide has a survey available to all companies that want to provide additional information about their environmental and social policies, programs and performance.