Data Collection Best Practices
Selecting the Metrics
We encourage you to pick as FEW metrics as possible! Markets often want to collect a long set of metrics, but often that makes the collection too difficult, especially when it means you will need a larger collection team. Or a market spends an enormous amount of time and energy to collect the data but then has nothing left in the tank for using that data at the end of the season.
Use the resources we offer to think about what you want to collect and why. Who is the audience for the data? Can one metric suit 2-3 audiences? Can you collect some this season and more data next season or next year? Don’t forget to reference your Metrics Audience Worksheet when thinking about metric selection.
FM Metrics is designed to calculate data for a representative season of 25 weeks, as that is the average length of a seasonal market’s “year” or is the median length of the “busy” season. If your market is a year-round market and you want this data to show the entire year’s impact, your market will collect two rounds of surveys or tallies: two to four times in the busy season, and two to four times in the quieter season. This will provide data that is representative of your market, and will ensure accurate estimates for the final metrics.
Preparing your Market
One of the most effective methods of preparation to maximize your data collection efforts is to prepare customers and vendors of your market. Tell them ahead of time that you’ll be collecting data. Tell them what you’re collecting, and why it’s important. Getting buy-in from your constituents on data collections will make your job easier, and your data more accurate.
Another equally important area of preparation is making sure you have data collection days staffed with adequate volunteers. Whether you’re only focusing on one metric, or you plan to have data collection days every market day, completing and disseminating your Data Collection Worksheet to all participants of your data collection team will help everyone be accountable for their part of the project.
Data Collection Instructions
Below are short guidance documents that speak to the methodologies that FM Metrics requires you follow as part of this program. If you feel that you need more guidance or instruction, please review the History and Context portion of this site, and be sure to attend the next Call Clinic to discuss with Darlene and other market participants from around the country.
Vendor information tells a powerful story about who is benefitting from selling at your market, and how their production practices are benefitting the community. Eleven simple questions, asked once a year add a significant amount of data on the impact your market has on the land, the local economy, and your local farmers.
Vendor data is added to FM Metrics through the use of a Vendor Profile. Ideally, the Vendor creates the profile themselves, but if this is not possible, a Market Manager can create a profile on their behalf. If the latter is true, an easy way to collect the information you need for their profile is to add the questions to your vendor application, or by provide the questions to your vendors as a standalone survey. The hardcopy version of Vendor Profile questions can be found in the Free Collection Resources area of the site.
Counting visitors entering your farmers market requires planning and adequate personnel. The number of data collectors needed depends on how many entry points your market has. Identify how many data collectors you’ll need, as well as how to prepare them for data collection by following the steps below.
Map Your Market: Create a map of your market by sketching its physical features. Include vendor booths, seating areas, buildings, and other structures. Then mark all entrances. Draw a line at each entrance, which classifies people as ‘in’ or ‘out’ of the market. It may be useful to draw that line in chalk onsite during your counting days. Identify any secondary entrances (between booths, through an alley, etc.) that could be easily blocked off on counting days. If you can move a garbage can, chairs or even use caution tape to block off a secondary entrance, it will help guide visitors to the primary entry points, and decrease the number of data collectors you’ll need.
Once you have identified the entrances, give each one a name (i.e., the street name, food court, etc.) so that everyone on your team is clear about which entrance is which. Ideally you will have one person assigned to each market entrance. However, if multiple entry points (primary and secondary) exist easily within the same field of vision, they can be monitored by one data collector. Create ‘zones of responsibility’ on your map which outline the areas (including one or multiple entrances) that can be monitored by one counter. Number the zones. The number of zones of responsibility on your map is also the number of data collectors you’ll need to perform a visitor count.
Sketch of a market map, with zones of responsibility and entry points identified.
On to the counting! We encourage markets to try to do Full Counts, meaning counting everyone entering all day, if possible. In many cases, however, this is not tenable given market resource constraints. The secondary method used within FM Metrics is a Sample Count; to count everyone entering the market for a set period of time. The second 20-minute window of each hour is the recommended window for counting. This method was chosen because it is generally agreed to be the most precise and the most doable for most markets.
Identify Your Counting Periods: Counts will be conducted for 20 minutes of each hour that the market is open. The first counting period should begin 20 minutes after the market opens, and the counting periods should remain consistent. So, if your market opens at 9:00am, counts will take place from 9:20am to 9:40am, 10:20am to 10:40am, and so on. Enter the time of the counting periods into the Visitor Count Record.
Prepare Collection Materials: Complete the top portion of the Visitor Count Record with your market’s name and map. Add any other information you’d like your data collectors to know, and print copies for each data collector. Provide data collectors a Visitor Count Record and a hand-held clicker. Hand-held clickers are inexpensive, subtle, and improve the accuracy of these counts. If no clickers are available, instruct data collectors to keep a tally of visitors on the back of their Visitor Count Record. If tallying by hand, data collectors may require clipboards.
Implementation: Assign one person to keep an eye on the clock, and to make sure that all entrances are staffed at the designated times. This person should also collect all Visitor Count Records at the end of the day. Give each data collector a Visitor Count Record, and highlight the instructions:
- Count only adults.
- Count individuals.
- Do not count visitors that re-enter.
- Do not count vendors.
- Record the total from each 20-minute counting period on the Visitor Count Record.
- If you can, set an alarm on your phone or watch for reminders to help maintain accuracy.
If you plan to use a method not discussed above, please discuss with FM Metrics staff before using, to ensure that it meets the minimum requirements for data collection as defined by researchers.
To learn more, read Counting Visitors at Markets in the FMC Resource Library.
Ideally, Visitor Surveys should be conducted on the same days as Visitor Counts. This makes it easier to know if the market collected a representative number of responses based on the attendance that day. If it’s not possible to complete both the Visitor Count and Visitor Surveys on the same day, then surveying the week following a Visitor Count is the next best approach. This will allow for a closer relationship between the attendance and the number of surveys.
Step 1: Identify Sample Size, Selection Interval and Number of Data Collectors
The sample size is the number of surveys needed to provide a reasonably representative amount of responses. The sample sizes listed in the table below will provide a 90% confidence level, with 5% margin of error. Try to reach your minimum sample size over the course of multiple market days, so you reach the widest variety of visitors.
Visitors who volunteer to take the survey, but aren’t in the selection interval should be allowed to take the survey, to support the desire for participation. The volunteered surveys can be separated out when entering the data (be sure to choose “volunteer” on these surveys).
To account for biases, data collectors should follow a systematic approach to selecting people to be interviewed. Depending on the sample size required at your market, a corresponding selection interval is provided. For example, if the visitor survey selection interval is 4, the data collector will survey every 4th adult that crosses the survey line. The table below gives recommended interval period and survey counts for a season.
Step 2: Map the Survey Line
Identify locations in the market where you can administer the survey. Near the market info booth, or a main entrance typically works well. Establish (or chalk) a line near the survey area. The data collectors will count the adults that walk past the line and ask every Nth adult to take the survey. If possible, have a table for shoppers to put down their items, or an umbrella for cover if the day is warm or misty.
Step 3: Prepare Materials
Customize the Visitor Survey template with your market name, the date of the survey, the target number of surveys to be completed and the selection interval that data collectors should use. Print enough surveys to fulfill your goal, as well as plenty of extras. Gather enough clipboards for your data collectors, pens and pencils.
Step 4: Implementation
Show the market map to your data collectors. Identify where they will administer the survey, how they will execute the selection interval (i.e., count adults that cross the ‘survey line’).
If possible, ask one of the data collectors to manage interval selection; they’ll approach every Nth market visitor to cross the survey line, ask that visitor if they’re willing to take a short survey, and direct them to a data collector with the surveys. This is especially important for larger markets.
Provide the data collectors with their materials, and go over the following instructions:
- Ask every Nth adult visitor who crosses the survey line to take a survey. A good opener is “Do you have a minute to help the market?”
- Ask the survey questions verbally, and record the answers on the survey form.
- If a visitor outside of the interval count volunteers to take the survey, circle ‘volunteer,’ on the survey. If a visitor does not want to complete the survey, asking them to participate does not count towards your minimum sample size. Identify the next respondent through the selection interval. In other words, don’t just ask the next adult to complete the survey.
Follow up with data collectors throughout the day to answer questions and hear about their experiences.
Tips for a Successful Survey
Conduct a few role plays with collectors before they begin so they can be confident from the start in their surveying. Positive energy makes surveying people at markets a fun job and increases their effectiveness.
Whatever the target number of surveys is, spread the number evenly across the hours the market is open. If all surveys are collected in the first hour, the answers will skew to a specific demographic and may not represent the entire spectrum of shoppers that the market attracts.
Vendor sales often go undocumented because of concerns about vendor privacy and difficulty in collecting the information, but it’s imperative that market managers have access to this information. Without sales information, you can’t make informed decisions about market operations, or accurately demonstrate your market’s impact on its vendors and local economy.
Many markets find that anonymous sales slips allows them access to vendor sales data, without making vendors uncomfortable. This document will outline the implementation of an anonymous vendor sales slip system, however, if you prefer to track the sales at the vendor level without anonymity, the process is easily adaptable — simply ask your vendors to include their name on their own Vendor Sales Slip.
Prepare the Materials
- Customize the Vendor Sales Slip to reflect your market’s name, and the payment methods used by vendors. Delete any unused payments methods in the Vendor Sales Slip Word document.
- Obtain a canister or box where vendors will be asked to deposit their completed Vendor Sales Slips each market day. Post a sign or label on the canister or box, so its purpose is clear. Vendors should feel confident that they’re putting their sales slip in the correct place, even if you aren’t nearby to confirm.
- Print a list of all vendors to place next to the canister or box, so vendors can check themselves off when they submit their slips.
- Print enough Vendor Sales Slips for all of the vendors who will attend your upcoming market day, with a couple extras for anyone who misplaces their slip.
Prepare Your Vendors
- If vendors are to provide you with accurate information, they’ll need to know that their data will be used responsibly, and will provide a real benefit to the market.
- Vendors should understand exactly how you will use their sales data. This is best accomplished in person at an annual vendor meeting, or in a market day “behind the table” chat; wherever vendors will feel comfortable asking questions. Explain how it will help make better management decisions (by knowing which promotions are working, and what other factors are impacting sales), and connect with new partners who can help support market operations (by demonstrating the economic value of the market’s work). Show an example of the type of summary report where the data will appear.
- Show examples of the Vendor Sales Slip, and describe the process. Vendors will complete the Vendor Sales Slip to the best of their ability at the end of every market day, fold the slip in half, place it into the marked box or canister at the market info booth, and check their name off of the list for the day.
- If your market uses tokens and a central point-of-sale device for processing credit, debit or federal nutrition benefits (we’ll call this your central terminal), explain that these sales will be recorded in the first column of the Vendor Sales Slip. If a vendor has their own point-of-sale device, and processes credit, debit, or EBT on an individual level, those sales will also be included in the first column the vendor sales slip where noted.
- The second column of the Vendor Sales Slip can be used for managing reimbursements for those markets that offer daily reimbursements for their vendors.
- Distribute the Vendor Sales Slips to each vendor at the beginning of every market day, by visiting each vendor. Remind them of the process and the need to turn the slip in before they leave for the day.
- Place the canister or box in an easy to view location (at the market booth or other central location), with a vendor list and pen or pencil easily accessible. It may be useful to walk the canister around the market at the end of the day, to collect sales slips, and remind vendors to complete them.
- If you operate a central terminal for credit, debit or federal nutrition benefit sales, reference the Central Terminal Transaction Report supplied by your service provider (most are available online), to enter your sales information from that market day into the Vendor Sales Slips.
- At the end of the market day, compile the total sales for the Market and enter these figures into the Per Market Day area of FM Metrics.
To find an average number of SNAP eligible goods available at the market, use the SNAP Eligible Goods Checklist, available in the Collection Resources area of the site. Each day that you complete the tally is represented by one column in the checklist (columns B, C, D, and E). The checklist may be completed directly in Excel, or printed out onto three 8.5 x 11 inch pages.
Starting at one end of the market, visit the first vendor’s booth or table. For every product type available, mark one tally in that product’s row, under the Data Collection Day One Column. If a vendor offers at least one type of SNAP-eligible product, they should be counted as a “Vendor Offering SNAP-Eligible Goods.” Add a tally for each of these vendors in row 15. Once you have marked a tally for every product offered for sale by the first vendor, move on to the next vendor’s booth, and repeat. Repeat for each vendor at the market. The Checklist also has a place to make note of the total number of vendors present, in column 14. To find this number, count as you go, or refer to market day attendance records.
If you complete the tally on paper, totals should be transferred into the SNAP Eligible Checklist in Excel, where calculations will automatically be completed. Scroll down to enter the total number of times a listed product appears for sale among all vendors on the data collection day. Once these numbers are entered into the Checklist in Excel, the workbook will calculate the metrics listed below. These should then be input into FM Metrics to track your responses over time.
- Average number of vendors offering SNAP eligible products per market day;
- Average number of of SNAP eligible products per market day;
- Total number of fruit and vegetable types offered for sale; and
- Percentage of Vendors Offering SNAP-Eligible Goods Per Market Day.