Sowing SEEDs for research at Oregon City Farmers Market

      Posted On: July 19, 2012

by Darlene Wolnik, Independent Trainer and Researcher for Public Markets & FMC Market Programs Advisor

Lara, a Portland public health student and intern for OCFM manages the table at the exit while Kristine completes a SEED survey. All respondents were invited to fill out a raffle ticket for a 25.00 shopping spree as a thank you for helping the market.

Oregon City-Where the Oregon Trail ended, the SEED trail begins for Oregon. The Oregon City Farmers Market (OCFM) was the site of the first official Oregon SEED study on June 23 of this year. SEED stands for Sticky Economic Evaluation Device and is the brainchild of marketumbrella.org (muo) in New Orleans– my hometown market folks. SEED takes a seven question economic survey of a (proper) sample size of shoppers and calculates pages of data that comes out in one report, packaged for city governments, other stakeholders and for markets themselves to use for years to come. The report offers economic data on the impact of the market on neighboring businesses quantifies sales at the market (without asking vendors to supply sales data) and gives a snapshot of the regional reach of a market through calculating its multiplier effect.

Jackie Hammond-Williams, the manager of both the Saturday and Wednesday OCFM markets approached me while I was in Oregon last. “What do you think of doing SEED at my markets?” she asked with a gleam in her eye and a purpose in her voice that made me think that the decision had already been made.

As we talked about why she felt she needed to do it, she echoed the very same reasons that had inspired marketumbrella.org to originally create the tool:

“Need data to make the case to the city about the market’s impact”
“A way to bring the board into the evaluation stuff”
“A baseline to track sales data”

Suzanne surveys market shoppers

This is how in mid June, I found myself heading southeast from Portland on a rainy early Saturday morning with Suzanne Briggs, Market SNAP/EBT expert, Hollywood (OR) Farmers Market founder, and FMC Technical Assistance Project Manager. I was to serve as the Study Director for the Saturday SEED study and Suzanne had graciously offered to volunteer as one of the pollsters so that she could experience the process.

Jackie had amassed a solid eight volunteers to act as the SEED team: two board members, two interns, three other valued volunteers, AND the market’s bookkeeper, all who often helped set up the market on Saturday mornings. In addition, there was Jackie’s market-suffering husband, Tim, who seems to be on the speed dial of every OCFM person to call on as needed. That group, along with Suzanne and I, made for an excellent team size for the study. The pollsters met at a nearby cafe for a Friday evening training. Training usually takes 30-45 minutes and usually consists of an overview of SEED’s purpose and going through the forms to explain the purpose. (Mostly, this meeting is a way for the group to meet and for the Study Director to see who should be assigned where.)

We would need 315 completed surveys to have the right sample size. That number had been calculated by clicking the attendance (1,778) at a Saturday market a few weeks prior, and using marketumbrella.org’s PDF of sample size found on their SEED site.

OCFM has always used the sample counting method of counting everyone in the market at preselected times (in this market’s case at 25 minutes after and 25 minutes before each hour) and then multiplying that number. The full count that was done two weeks prior to our visit, with people stationed at the exits and counting every adult entering, had resulted in a number higher than previously counted on similar days. One explanation may be that they do not usually count the music entrance/area. So there you have it– one good reason to at least alternate the sample counting method with full counting method.

Based on the end goal of 315 completed surveys, we needed to conduct an average of 63 each hour during the market hours of 9 to 2. That meant stationing three to four people at key exit points and collecting answers to these seven questions:

How often do you come to this market?
How much did you spend today at the market?
Is the market the primary reason you came to this area today?
If yes, will you spend any money at neighboring businesses?
If so, how much?
In what zip code do you live?
(Gender)

Like many market managers, Jackie had many jobs on her plate in springtime, and we had not found the time up until my visit to create her SEED account online. I brought along my iPad so that her bookkeeper/data entry person could help me set the account up.

The account has three parts-

1) Organizational information: Type of incorporation (if any), number of staff (if any), operating budget estimate (if any), and so on. A bookkeeper or a board member should know or have access to this information.

2) The Market Portrait: The Portrait outlines what marketumbrella.org calls a market’s 4 Ps: Product mix, place, people (shoppers) and procedures. The Portrait is part of every SEED report done for that market location; it gives stakeholders a sense of the market’s structure and history. Because the Portrait report can stand alone (independent of a SEED study) and thus can be used in other ways– as an introductory handout for new vendors, as part of that market’s annual report, or simply as an institutional record of each market location’s evolution.

3) SEED survey data entry: All of the surveys collected at the market must be entered into the muo portal online. Later on, individual market studies can be combined for a more comprehensive report that includes all or some of an organization’s markets. For example, if an organization runs three weekday markets and a Saturday market, it might want a report of all four markets or just the weekday markets. Because all of the information remains available to the market through its marketshare account, reports can be accumulated, and anyone with access can download their reports as needed.

In rain that at times was best described as “driving,” energetic pollsters were stationed under tents at the two busiest exits, asking exiting shoppers the survey questions. As this process got started, we set up the free SEED account while I also watched the ebb and flow of the pollsters. The study director (that’s me) needs to keep an eye on their team in case they need to adjust for an overflowing exit point, or move a shy pollster closer to an outgoing colleague to spread the energy or send someone to cover for a break.

An item in the previous week’s market e-newsletter alerted shoppers about the research effort and requested their participation in the survey process. Using a similar request, pollsters approach potential respondents with “do you have a minute to answer six questions to help the market?” Shoppers do want to help the market, and will often gladly stop to answer a few questions, especially when there is a table on which to set their bags, and some overhead coverage to shade from sun or sprinkle. And, not surprisingly, Jackie cajoled, pointed, and sometimes walked her regulars to the tents on their way out.

Jackie also made sure her vendors knew about the survey ahead of time. Market staff or regular volunteers should always advise vendors directly about unusual activities like surveys to be conducted at market. Vendors can be your best allies during a shopper survey, as many have achieved a significant level of trust with customers, meaning that, their customers will literally follow their pointed finger in the direction of the survey tent.

The day went well, although the lower attendance due to rain cast doubt on our ability to meet our sample size quota. However, as Jackie predicted, the sun’s arrival at 11 o’clock ushered in the bulk of shoppers, and our by then seasoned pollsters got many respondents and even exceeded the projected hourly numbers. With such an efficient  team,  felt we could get some optional Forager forms filled out, too. The Saturday market is set up in a light industrial parking lot a block or more away from other street activity, so we used the Forager form to gather data from the vendors, with pollsters careful not to interrupt the flow of business. Jackie advised the Board member assigned to forage what times to expect a lull and pointed him to those vendors who usually brought help, those that she suspected could spare some time, and of course those that she knows just love to talk!

By 1:30, we had exceeded our of shopper survey quote by  37. Soon after, we let the pollsters stop collecting to begin breaking down the market. Everyone seemed to enjoy their time on clipboard duty, thrilled that they had accomplished this feat for their market. I collected the last of the sheets, promising to give them back to the market before I left town. I wanted to make sure that the data had been documented correctly and highlight any outliers. Someone buying half a cow and spending hundreds of dollars would be considered an outlier, assuming that’s not a weekly occurrence for them. However, those unique large purchases could definitely be used as an example in the press releases!

Below is the page where one enters the data from the pollster forms.  Each page allows for the entry of 10 surveys.

SEED data entry page

SEED data entry should be conducted by one or two people who can take the time to get it all entered as quickly as possible. In this way, pollsters can be contacted sooner rather than later if a question arises and requires clarification. Doing so, of course, requires that you ask pollsters to put their names at the bottoms of their forms– the one thing I did forget! Lucky for me, I didn’t have any legibility problems.

SEED pollster form

In a few hours, I completed the data entry for the Saturday market. I took the surveys back to Jackie on Monday evening to discuss the results, meeting not only with her, but with Tim, Suzanne, and the president of the Oregon Farmers Market Association, Rebecca Landis along with her intern Brett.

The discussion centered on the most important pieces of SEED and using the tool as designed in order for its data to be most accurate and useful. In other words, collecting the proper sample size on one day, asking each of the seven questions on the form, and watching for outliers are all quite necessary.

Looking at the 12-page report over dinner on Jackie’s back porch, we each found nuggets of data that interested us: for example, the zip code list

Suzanne, Darlene, and Oregon City Farmers Market volunteer, Lynne, conducting SEED surveys on a rainy market day at Oregon City Farmers Market.

of visitors and how much each area contributed to the overall economic picture; we could see how that data would be helpful to community partners and for the market management to use for outreach. Another item we discussed was the average sale; another was the number of weekly versus first time shoppers. Then, we had a discussion about methodology and multiplier effect that ended with the usual vendor success storytelling among market managers…

Knowing that every market has not decided how to use a tool like SEED yet, I also briefly shared the indicator work that a small team at FMC had started in 2011. The Indicator Matrix will be a simple yet user-friendly matrix of appropriate data collection points for markets to sift through. It will be the collection of the best questions in which to evaluate markets in economic, social, human and natural terms, such as those found in the SEED tool. The Indicator Matrix would make it possible for a market and its partners to begin disciplined evaluation by deciding on a single or a concerted set of questions to measure success of a single project or a facet of the market. Once finalized, the questions will measure the impact on market members, (whether its for its vendors, shoppers or the larger community), calculating how much time and effort it would take to collect that data, and who the potential audience for that data would be.

In other words, this Indicator Matrix is designed with the goal of cultivating a culture of data collection at every market. Once on that path, using SEED and tools like it would happen in every corner of the market world.

See a sample of the FMC Indicator Matrix here.

As for Oregon City, I think my most helpful role is still to come. The analysis and dissemination of the report really should be set up as a series of campaigns – how to share this story with the Board and the vendors, then with the community partners and so on. SEED reports are so full of data that each part can be mined and communicated for months to come.

I hope Oregon City’s SEED study leads to many more for Oregon. I’m glad I was around for the first.

Here’s a teaser from the first Oregon SEED report:

Using the Sticky Economy Evaluation Device (SEED) methodology, marketumbrella.org is pleased to report that the Oregon City Farmers Market-Saturday has an annual combined economic impact of $3,757,907.56 on its vendors, host neighborhood, and surrounding region. Check with OCFM for more information about their SEED report and check out the FMC evaluation tools and resources online.