Data Collection Best Practices

Vendor information tells a powerful story about who is benefitting from selling at your market, and how their production practices are benefitting the community. Eleven simple questions, asked once a year add a significant amount of data on the impact your market has on the land, the local economy, and your local farmers.

Vendor data is added to FM Metrics through the use of a Vendor Profile. Ideally, the Vendor creates the profile themselves, but if this is not possible, a Market Manager can create a profile on their behalf. If the latter is true, an easy way to collect the information you need for their profile is to add the questions to your vendor application, or by provide the questions to your vendors as a standalone survey. The hardcopy version of Vendor Profile questions can be found in the Free Collection Resources area of the site. 

Sketch of a market map, with zones of responsibility and entry points identified.

On to the counting!  We encourage markets to try to do Full Counts, meaning counting everyone entering all day, if possible. In many cases, however, this is not tenable given market resource constraints. The secondary method used within FM Metrics is a Sample Count; to count everyone entering the market for a set period of time. The second 20-minute window of each hour is the recommended window for counting. This method was chosen because it is generally agreed to be the most precise and the most doable for most markets.

Identify Your Counting Periods: Counts will be conducted for 20 minutes of each hour that the market is open. The first counting period should begin 20 minutes after the market opens, and the counting periods should remain consistent. So, if your market opens at 9:00am, counts will take place from 9:20am to 9:40am, 10:20am to 10:40am, and so on. Enter the time of the counting periods into the Visitor Count Record.

Prepare Collection Materials: Complete the top portion of the Visitor Count Record with your market’s name and map. Add any other information you’d like your data collectors to know, and print copies for each data collector. Provide data collectors a Visitor Count Record and a hand-held clicker. Hand-held clickers are inexpensive, subtle, and improve the accuracy of these counts. If no clickers are available, instruct data collectors to keep a tally of visitors on the back of their Visitor Count Record. If tallying by hand, data collectors may require clipboards.

Implementation: Assign one person to keep an eye on the clock, and to make sure that all entrances are staffed at the designated times. This person should also collect all Visitor Count Records at the end of the day. Give each data collector a Visitor Count Record, and highlight the instructions:

• Count only adults.
• Count individuals.
• Do not count visitors that re-enter.
• Do not count vendors.
• Record the total from each 20-minute counting period on the Visitor Count Record.
• If you can, set an alarm on your phone or watch for reminders to help maintain accuracy.

If you plan to use a method not discussed above, please discuss with FM Metrics staff before using, to ensure that it meets the minimum requirements for data collection as defined by researchers.

To learn more, read Counting Visitors at Markets in the FMC Resource Library.

Ideally, Visitor Surveys should be conducted on the same days as Visitor Counts. This makes it easier to know if the market collected a representative number of responses based on the attendance that day. If it’s not possible to complete both the Visitor Count and Visitor Surveys on the same day, then surveying the week following a Visitor Count is the next best approach. This will allow for a closer relationship between the attendance and the number of surveys.

Step 1: Identify Sample Size, Selection Interval and Number of Data Collectors
The sample size is the number of surveys needed to provide a reasonably representative amount of responses. The sample sizes listed in the table below will provide a 90% confidence level, with 5% margin of error. Try to reach your minimum sample size over the course of multiple market days, so you reach the widest variety of visitors.

Visitors who volunteer to take the survey, but aren’t in the selection interval should be allowed to take the survey, to support the desire for participation. The volunteered surveys can be separated out when entering the data (be sure to choose “volunteer” on these surveys).

To account for biases, data collectors should follow a systematic approach to selecting people to be interviewed. Depending on the sample size required at your market, a corresponding selection interval is provided. For example, if the visitor survey selection interval is 4, the data collector will survey every 4th adult that crosses the survey line. The table below gives recommended interval period and survey counts for a season.

 

Step 2: Map the Survey Line
Identify locations in the market where you can administer the survey. Near the market info booth, or a main entrance typically works well. Establish (or chalk) a line near the survey area. The data collectors will count the adults that walk past the line and ask every Nth adult to take the survey. If possible, have a table for shoppers to put down their items, or an umbrella for cover if the day is warm or misty.

Step 3: Prepare Materials
Customize the Visitor Survey template with your market name, the date of the survey, the target number of surveys to be completed and the selection interval that data collectors should use. Print enough surveys to fulfill your goal, as well as plenty of extras. Gather enough clipboards for your data collectors, pens and pencils.

 

Step 4: Implementation
Show the market map to your data collectors. Identify where they will administer the survey, how they will execute the selection interval (i.e., count adults that cross the ‘survey line’).

If possible, ask one of the data collectors to manage interval selection; they’ll approach every Nth market visitor to cross the survey line, ask that visitor if they’re willing to take a short survey, and direct them to a data collector with the surveys. This is especially important for larger markets.

Provide the data collectors with their materials, and go over the following instructions:

1. Ask every Nth adult visitor who crosses the survey line to take a survey. A good opener is “Do you have a minute to help the market?”
2. Ask the survey questions verbally, and record the answers on the survey form.
3. If a visitor outside of the interval count volunteers to take the survey, circle ‘volunteer,’ on the survey. If a visitor does not want to complete the survey, asking them to participate does not count towards your minimum sample size. Identify the next respondent through the selection interval. In other words, don’t just ask the next adult to complete the survey.

Follow up with data collectors throughout the day to answer questions and hear about their experiences.

 

Tips for a Successful Survey
Conduct a few role plays with collectors before they begin so they can be confident from the start in their surveying. Positive energy makes surveying people at markets a fun job and increases their effectiveness.

Whatever the target number of surveys is, spread the number evenly across the hours the market is open. If all surveys are collected in the first hour, the answers will skew to a specific demographic and may not represent the entire spectrum of shoppers that the market attracts.

Vendor sales often go undocumented because of concerns about vendor privacy and difficulty in collecting the information, but it’s imperative that market managers have access to this information. Without sales information, you can’t make informed decisions about market operations, or accurately demonstrate your market’s impact on its vendors and local economy.

Many markets find that anonymous sales slips allows them access to vendor sales data, without making vendors uncomfortable. This document will outline the implementation of an anonymous vendor sales slip system, however, if you prefer to track the sales at the vendor level without anonymity, the process is easily adaptable — simply  ask your vendors to include their name on their own Vendor Sales Slip.

Prepare the Materials

• Customize the Vendor Sales Slip to reflect your market’s name, and the payment methods used by vendors. Delete any unused payments methods in the Vendor Sales Slip Word document.
•  Obtain a canister or box where vendors will be asked to deposit their completed Vendor Sales Slips each market day. Post a sign or label on the canister or box, so its purpose is clear. Vendors should feel confident that they’re putting their sales slip in the correct place, even if you aren’t nearby to confirm.
• Print a list of all vendors to place next to the canister or box, so vendors can check themselves off when they submit their slips.
• Print enough Vendor Sales Slips for all of the vendors who will attend your upcoming market day, with a couple extras for anyone who misplaces their slip.

Prepare Your Vendors

•  If vendors are to provide you with accurate information, they’ll need to know that their data will be used responsibly, and will provide a real benefit to the market.
• Vendors should understand exactly how you will use their sales data. This is best accomplished in person at an annual vendor meeting, or in a market day “behind the table” chat; wherever vendors will feel comfortable asking questions. Explain how it will help make better management decisions (by knowing which promotions are working, and what other factors are impacting sales), and connect with new partners who can help support market operations (by demonstrating the economic value of the market’s work). Show an example of the type of summary report where the data will appear.
• Show examples of the Vendor Sales Slip, and describe the process. Vendors will complete the Vendor Sales Slip to the best of their ability at the end of every market day, fold the slip in half, place it into the marked box or canister at the market info booth, and check their name off of the list for the day.
• If your market uses tokens and a central point-of-sale device for processing credit, debit or federal nutrition benefits (we’ll call this your central terminal), explain that these sales will be recorded in the first column of the Vendor Sales Slip. If a vendor has their own point-of-sale device, and processes credit, debit, or EBT on an individual level, those sales will also be included in the first column the vendor sales slip where noted.
• The second column of the Vendor Sales Slip can be used for managing reimbursements for those markets that offer daily reimbursements for their vendors.

Implementation

• Distribute the Vendor Sales Slips to each vendor at the beginning of every market day, by visiting each vendor. Remind them of the process and the need to turn the slip in before they leave for the day.
• Place the canister or box in an easy to view location (at the market booth or other central location), with a vendor list and pen or pencil easily accessible. It may be useful to walk the canister around the market at the end of the day, to collect sales slips, and remind vendors to complete them.
• If you operate a central terminal for credit, debit or federal nutrition benefit sales, reference the Central Terminal Transaction Report supplied by your service provider (most are available online), to enter your sales information from that market day into the Vendor Sales Slips.
• At the end of the market day, compile the total sales for the Market and enter these figures into the Per Market Day area of FM Metrics.

More and more states are starting to actively collect data on prices at farmers markets. This information is invaluable for farmers, customers and the USDA, as it contributes to the availability of crop insurance for direct marketed specialty crops. FMC is working with some of these states to help define a standard method for collecting and sharing this data. Until that method has been identified, this section is under construction. Check back soon.

To find an average number of SNAP eligible goods available at the market, use the SNAP Eligible Goods Checklist, available in the Collection Resources area of the site. Each day that you complete the tally is represented by one column in the checklist (columns B, C, D, and E). The checklist may be completed directly in Excel, or printed out onto three 8.5 x 11 inch pages.

Starting at one end of the market, visit the first vendor’s booth or table. For every product type available, mark one tally in that product’s row, under the Data Collection Day One Column. If a vendor offers at least one type of SNAP-eligible product, they should be counted as a “Vendor Offering SNAP-Eligible Goods.” Add a tally for each of these vendors in row 15. Once you have marked a tally for every product offered for sale by the first vendor, move on to the next vendor’s booth, and repeat. Repeat for each vendor at the market. The Checklist also has a place to make note of the total number of vendors present, in column 14. To find this number, count as you go, or refer to market day attendance records.

If you complete the tally on paper, totals should be transferred into the SNAP Eligible Checklist in Excel, where calculations will automatically be completed.  Scroll down to enter the total number of times a listed product appears for sale among all vendors on the data collection day. Once these numbers are entered into the Checklist in Excel, the workbook will calculate the metrics listed below. These should then be input into FM Metrics to track your responses over time.

• Average number of vendors offering SNAP eligible products per market day;
• Average number of of SNAP eligible products per market day;
• Total number of fruit and vegetable types offered for sale; and
• Percentage of Vendors Offering SNAP-Eligible Goods Per Market Day.