Building a Matrix for Market Measurement: An Update on FMC’s Indicator Project
Posted On: April 18, 2012
By now, 7,175 is a familiar number. It is the recorded number of U.S. farmers markets operating in 2011, according to the USDA Agricultural Marketing Service. In every one of those 7,175 markets, someone – maybe a group of farmers, nearby business owners, hopeful shoppers, a well-meaning government agencies eager for economic or social activity – decided to create a place where regional food could be sold and shared. Those varied origins and stakeholders could lead to at least 7,175 different ways in which success could be measured in food systems. What to measure and how to measure market activity has long been a point of anxiety for market organizers with limited capacity.
As markets begin, the buzz of activity at the market itself is often the easiest and earliest gauge of success. Organizers can count the quantity of both vendors and shoppers that come regularly. Even in the collection of those numbers, however, there is a great deal of guessing. What is the best method for a busy organizer to know how many shoppers stream in and out of those hidden passageways between tents? Or how much and what kinds of products they might actually be purchasing? After shopper and vendor numbers are established, more expansive measurements might be needed, such as the increase in healthy food production or the addition of jobs. Perhaps the market was organized to increase farmland use, or its partners are interested in the quality of educational events offered, or the degree to which neighboring businesses (or the city’s parking meters!) benefit from the traffic brought to the market area. Each of these questions that lead to data collection at the market is called an indicator in the parlance of funders or researchers. For a market searching for ways to define success of a funded project or to convince their municipality that market activity matters, the first step is to understand which indicator should be chosen and how to use it. That decision would be based on answers to the following questions:
- Which audience wants and will use the data: a municipality, a partner organization, or the market itself in an annual report?
- Which group within a market community you want to measure impact on: the farmers, shoppers, or the larger community (using the “triple bottom line” that FMC sets as a framework)?
- What type of benefit is added to the market community across the spectrums of economic, social, human, or natural capital? Those types of benefits began to be identified and tested by some flagship markets and support organizations in the 1990s, including New Orleans-based public market organization marketumbrella.org.
The working definitions of each might be helpful here:
Economic impacts (money) are defined by measures such as adding vendor sales, creating jobs or adding sales to neighboring businesses.
Social impacts (trust, bridging and bonding) are defined by measures such as friends meeting at the market, families spending time together, youth volunteering, or vendors helping each other or donating food to local social service agencies.
Human impacts (knowledge, skills) are defined by measures such as shoppers learning new recipes (or about how certain crops are grown), vendors learning new languages to better communicate with shoppers, or community partners learning about market shopper behavior.
Natural (or ecological) impacts would include land use patterns, acres preserved for productive farmland, distance and length of time from harvest to final consumer, crop diversity, and certified and non-certified sustainability, soil/water conservation practices.
In 2011, the Farmers Market Coalition worked with technical assistance providers Suzanne Briggs and myself to assist communities in four states that were participating in the CDC’s Communities Putting Prevention to Work program. Our team worked on templates for those programs to assess their success in implementing or expanding SNAP programs in their markets. Through that technical assistance, a matrix of known indicators was born for markets to select what to measure based on the audience and time and resources allotted for the project. The indicators listed were ones already being used by markets to measure themselves or were included from suggestions by partner organizations.
Known indicators came from a variety of places. Most notably, from research such as marketumbrella.org’s trans•act project, which resulted in their tool SEED (Sticky Economic Evaluation Device), measuring economic activity at the market and the multiplied effect on the surrounding community. That tool then led to their pilot of NEED (Neighborhood Equity Evaluation Device) , a project on which I worked on at the time, and on which the Farmers Market Coalition also partnered, designed to measure social capital. FEED (Food Equity Evaluation Device), aims to measure nutrition/health impact. Both NEED and FEED remain in pilot form, while SEED is available online for markets to use freely at this point.
Research continues at marketumbrella.org to search for and test other ways to measure the addition of capital in markets. Organizations such as the Wholesome Wave Foundation, Project for Public Spaces, and Community Food Security Coalition have also pioneered data collection in markets/local food systems on health, social capital, and food equity, respectively. In every region in the U.S., public health teams work energetically on food access and quality at markets, collecting data points which have also been incorporated into the list of indicators in the current draft of the FMC matrix.
As momentum grows behind farmers markets, FMC sees an imperative to align the vision behind local food systems with indicators for measuring progress. If local food systems are to continually improve their service to stakeholders, practitioners should be empowered with a menu of indicators to benchmark and communicate their impacts to community partners and decision-makers. And yet, those measurements must be used in a disciplined study, one that meets other sectors’ expectations for proper methodology and sample sizes without major disruption to the economic activity of the market.
It is also clear to FMC that the largest challenge to markets collecting their own meaningful data is their very efficiency: they rarely have the staff size to take on an evaluation study or the experience at the beginning of their projects to sit and think through evaluation. The matrix would be used at the beginning of a project, or the beginning of the season, when partners would select the appropriate indicators with all parties understanding what, who, when, and how the data will be collected and then communicated. This is also why the matrix contains suggested time allotments needed for the entire study. It is also why the matrix is set up as “a la carte”; each indicator stands on its own as a data collection point. Of course, the more data collected, and the more impacts studied, the better.
With additional support, these indicators can be fleshed out, piloted, revised, and made into a user-friendly interface for use by Farmers Market
The rolling hills of the Morven Summer Institute
Coalition members and adapted as needed for other sectors of local food. My research on behalf of FMC has continued in 2012 to further the refinement and collection of data for the indicator work. This summer, FMC will be collaborating with the University of Virginia on a two-week course called Farmers Markets and Applied Food Systems Research (PLAC 5501) as part of the Morven Summer Institute. The lead course instructors will be Dr. Paul Freedman, in the Department of Politics and Dr. Tanya Denckla Cobb in the Department of Urban & Environmental Planning. Other partners exploring this indicator work with FMC already include markets in North Carolina and food justice organizations searching for systemic ways to build grassroots capacity in food systems across the country.
In all cases, measurement needs are being addressed in various ways by food system and social justice organizations in every region. FMC is planning to continue to strategize and connect with a multitude of partners interested in building or piloting the matrix. Expect more details on measurement soon and feel free to contact me with ideas on measurement and for information on the tools that already exist for markets.
Examples of 3 draft indicators
Youth working with vendors
Impact: Economic (Vendors), Social, Human (Community)
Audience: Social service agencies, youth organizations, Market vendors
Methodology: Intercept surveys (Vendor)
Capacity: Survey team/1 market day
Number of farmers transitioning to organic methods
Impact: Social, Human (Vendor), Natural (Community)
Audience: Agricultural organizations, Environmental organizations
Methodology: Intercept surveys (Vendor)
Capacity: Survey team/1 market day
Number of educational cooking demonstrations staged
Impact: Social, Human (Shopper)
Audience: Public health, Educators
Methodology: Office research
Capacity: 1 person < 2 hours
