Collect data for AI training and evaluation

Why can't AI read the signs in our neighborhood?

What if a development team cannot travel to a region missing from its data? A story about collecting the scenes it needs with the local community.

HUMINT / FIELD NOTESAI TEAMS · RESEARCH INSTITUTIONS

The questions behind the headline

Suppose a model reads signs well on sunny days but repeatedly fails in rainy alleyways. Before collecting more images, look at which regions and conditions are missing from the data.

The person who knows the scene best may be someone who walks past it every day. Specify the time, weather and subject, then collect images with labels, collection times and usage rights so the data can be reviewed.

Clues from a real case

GLOBAL / COMMUNITY DATA · Published 2026-09-18

Common Voice expands community language data

Mozilla released Common Voice Scripted Speech v27.0 and Spontaneous Speech v5.0. The read-speech collection covers 295 languages, while the spontaneous-speech collection added six languages, including Chinese and Swahili. The release illustrates how direct community contributions can fill gaps in language data.

Mozilla · Common Voice · Read the source

What to ask people on the ground

The proposal below adapts a published example to HUMINT. It does not imply that the referenced organization uses HUMINT.

Apply the model of local contributors creating and reviewing data to image collection. Request images of signs and surroundings in daylight, shade and rain across three underrepresented regions. Exclude faces, license plates and private spaces, and link each file to its labels in a CSV.

Requirement
Collect sign and environmental images in the specified region under daylight, shade and rain conditions. Record their categories and usage rights.

Specify the taxonomy, file requirements, collection conditions and deduplication criteria. Link images and CSV labels with shared identifiers, and state collection consent, usage rights and personal-data exclusions in the requirement.

Follow-up questions

Which class and capture conditions apply to each image, and how are files linked to labels?
What establishes collection consent and usage rights, and does the material duplicate an earlier submission?

Evidence to request with the answer

Images with confirmed collection consent and usage rights, plus CSV files linking classification labels and collection dates.

JPG
PNG
WebP
CSV

View the collection fields

Fields to record

Include these fields in the report or CSV to connect observations with supporting files.

image_id
filename
label
region_code
collected_at
lighting
weather
rights_basis

How to read the limits of the evidence

Review image-label consistency, duplicates, collection conditions and evidence of usage rights. Organize approved material into training and evaluation sets as appropriate; accepting a submission on the platform is not an automatic quality certification.

Check file-label identifiers, classification criteria, collection conditions and evidence of usage rights together.
Review near-duplicate images and personal data, and split training and evaluation material by contributor, region and time to reduce leakage.

Review the collection and split it into training and evaluation sets to check which regions and conditions still cause mistakes. Small observations from the field become data that helps fill AI's gaps.

Participation and rewards

Separate the public overview from restricted details and set access levels and approval conditions. Deposit the reward and fee reserve, then review each submission package and its proposed HRS reward before approving recipients. Participation can be rated independently of payout selection.

FROM CONTEXT TO ACTION

Ask the questions that need a local answer.

Define your questions, deliverable criteria, access level and HRS reward, then publish a requirement.

Create a requirement