Farmers Market Metrics: The Research Process
By: Darlene Wolnik Posted On: July 21, 2014
As the research lead for the Farmers Market Coalition’s Market Metrics work, I decided to offer some background about the FMC/University of Wisconsin AFRI-funded research that is happening over the next three years.
Here’s what we know: Almost all markets and their partners already collect data. It may be the SNAP tally sheet at the end of the market, which tells how many benefit program users came that day or how many tokens they used. It might be clicking how many visitors came to the market or how many products were available to purchase, or the sales reported by the market’s vendors. The reality is that there is actually a great deal of information available about markets. There are even a few excellent tools to do some collection and reporting, such as Market Umbrella’s SEED tool, Projects for Public Spaces Placemaking audits, Demonstrating Value’s market toolkit, Wholesome Wave’s incentive tracking portal, or state-level market tracking such as West Virginia’s data collection project and so on.
And a few markets and their market partners are great at analyzing how well they use their funding or their staff levels or are adept at showing the changes in their community. So there are already dozens of reports available that detail the impact of markets on their shoppers and their vendors, on the neighboring businesses and on the larger community. Check out the FMC Resource Library for a representation of those.
Data (and some tools) surely exist. So what’s the problem?
Data is often collected across different time spans, using different definitions, rarely uses comparable analysis or matches the output with the specific market profile. And the market community itself is often left out of the design of the collection and the reports.
How that data was collected: what questions were asked, and of whom and when and how well designed it was to ensure no bias or assumptions colored the results. Understaffed markets organizations are often uneasy in participating in the actual collection, often leaving it to external partners; as a result, the raw data is often not shared with the markets.
What type of impact does that data really show? Is it an internal benchmark or goal specifically for that market community to learn from or is there an external change that can affect policy? Can it be compared to other sectors or other markets in the food system? Does it show a wide spectrum of benefits, meaning more than just economic? Markets rely on external partners to provide that analysis, but often that analysis represents only the change that partner is primarily interested in.
Forty-plus years after the market revival began, most markets expect to have many impacts: curating local agriculture sales, acting as the town square, encouraging knowledge transfer, expanding new businesses and new initiatives, and offering healthier options for all, including for at-risk citizens. Markets often have other less universal goals too, such as welcoming immigrant communities, or supporting non-market initiatives. However, reports that show the multiple benefits of markets the unique market characteristics that deliver those benefits are often only anecdotal if they exist at all.
Therefore, any measurement of success for markets across the US needs to identify universal indicators for any or all of these changes, to allow markets to compare that change to others while still making room for each market and its partners to tell its own unique story. It’s also important not to duplicate the tools already available and to support existing data collection initiatives.
Where to start? For the Farmers Market Coalition and the University of Wisconsin team, it seemed best to start at the market level by asking these questions:
- What if someone collected the best metrics now available and refined them for markets and market partners to use?
- What if markets took the lead in deciding which of those metrics to use (with some help) based on their immediate issues or to help with their long-term planning?
- What resources are needed to help largely volunteer entities know how to properly collect or supervise the collection of data and to use that data to make their case for expanded capacity?
- And what if those results (indicators) were collected and shared at the national level so everyone could see the data, compare results and decide what else could be measured?
From June 2014 to 2016, FMC and University of Wisconsin will be conducting applied research on these questions at nine market sites in three regions. The focus of this research will be to uncover the barriers and challenges that keep markets from collecting and using data as well as refining metrics for market communities to begin to make their case on the potential multiple benefits of markets.
The three years will be an iterative process and therefore, is designed to encourage feedback and input from as many voices as possible. The nine markets will receive stipends, support from the project team in selecting the metrics and in sharing the indicators. Other markets (not selected as pilot sites) will also be invited to offer feedback on some of the activities, and participate in some of the educational webinars to offer training and resources for the nine markets. Market partners’ input on metrics and analysis will be solicited and shared throughout. Finally, the draft reports and info-graphics will be uploaded on FMC’s website for feedback and invitation for funders to assist with future iterations. You can already view prototype reports uploaded on the FMC Resource Library that we completed in June of 2014. These reports were for markets that uploaded 2013 data so that the FMM team could gain feedback on the use of those prototype reports; that quick round of prototypes will assist in the reporting phase of this 3-year research.
At the end of the three years, there will be a beginning set of metrics, resources to address data collection barriers at the grassroots level and drafts of the reports using the data collected by the nine markets. All of the data collected will be given back to the markets in spreadsheets, in info graphic-style reports and made available to researchers and partners to continue to expand reasonable and disciplined data collection and analysis.
Obviously, many questions remain and more pilots will certainly be necessary to build a true system for evaluation of markets. Be patient with us in this opening round and please check back here to find invitations for shared learning on this work or to read the updates. Feel free to share ideas, criticism or news of similar projects with FMC or to offer ideas for future rounds of research and pilots.
Keep my email available to let me know any questions that you might still have.
Email: darlene@ farmersmarketcoalition.org
FMM Research Lead
New Orleans LA