STEAMHOUSE PILOT MEASUREMENT TOOLKIT
Data Collection Instruments for Program Implementation & Evaluation
Version 1.0 | December 2025
EXECUTIVE SUMMARY
Purpose of This Toolkit
This toolkit provides standardized instruments for collecting data from steamHouse pilot implementations. The instruments are designed to be:
Useful for internal learning — even without formal research partners
Methodologically sound — won't compromise future rigorous evaluation
Attractive to researchers — demonstrates evaluation readiness
Feasible for partners — minimal burden on participants and mentors
What's Included
InstrumentPurposeFrequencyTime Required1. Participant Intake FormBaseline demographics and contextEntry (once)5-7 minutes2. Mentor Weekly Fidelity LogImplementation trackingWeekly3-5 minutes3. Monthly Participant ReflectionMetacognitive development evidenceMonthly5-10 minutes4. Development Marker Self-AssessmentProgress tracking on 53 markersEntry + Exit15-20 minutes5. Pilot Partner AgreementData collection expectationsEnrollmentN/A
Data Collection Packages
Minimal Package (ANY partner):
Intake Form + Exit Survey + Attendance tracking
Total participant burden: ~15 minutes total
Standard Package (committed partners):
All of Minimal, plus Monthly Reflection + Entry/Exit Self-Assessment
Total participant burden: ~60-90 minutes over full pilot
Research-Ready Package (high-capacity partners):
All of Standard, plus full Mentor observations + optional parent survey
Total participant burden: ~90-120 minutes over full pilot
INSTRUMENT 1: PARTICIPANT INTAKE FORM
Purpose
Collect baseline information about participants at enrollment. Essential for understanding who is being served and enabling later analysis by demographics, prior experience, and context.
Administration
When: First session or before first session
How: Paper form, online survey, or mentor-assisted completion
Who completes: Participant (ages 12+) or parent/guardian (ages 8-11)
Time: 5-7 minutes
STEAMHOUSE PARTICIPANT INTAKE FORM
Program Information (completed by mentor)
FieldResponseSite/Program Name_________________________Activity Type_________________________Mentor Name_________________________Enrollment Date//________Participant ID_________________________
Note: Participant ID should be unique but not include name. Suggested format: Site abbreviation + number (e.g., FLL-CO-001)
SECTION A: About You
A1. Name (for site records only; will not be used in aggregate data)
First: _________________________ Last: _________________________
A2. Date of Birth
//________ (MM/DD/YYYY)
A3. Age at enrollment: _______ years
A4. Grade level (if applicable):
3rd grade or below
4th grade
5th grade
6th grade
7th grade
8th grade
9th grade
10th grade
11th grade
12th grade
Not currently in school
Other: _________________________
A5. Gender:
Female
Male
Non-binary
Prefer to self-describe: _________________________
Prefer not to say
A6. Race/Ethnicity (check all that apply):
American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or Pacific Islander
White
Other: _________________________
Prefer not to say
SECTION B: Your Experience
B1. Have you participated in this type of activity before? (e.g., if this is FLL, have you done robotics before?)
No, this is my first time
Yes, for less than 1 year
Yes, for 1-2 years
Yes, for 3+ years
B2. Have you participated in steamHouse programs before?
No, this is my first steamHouse experience
Yes, for less than 1 year
Yes, for 1-2 years
Yes, for 3+ years
B3. How did you hear about this program? (check all that apply)
Parent/guardian
Friend
School announcement
Community organization
Social media
Other: _________________________
B4. Why are you joining this program? (check your TOP 3 reasons)
To learn new skills
To be with friends
To try something new
Parent/guardian wanted me to
To compete/perform
To be part of a team
To work on interesting projects
To meet new people
Other: _________________________
SECTION C: About Your Household (Parent/Guardian Completes)
C1. Parent/Guardian completing this form:
Name: _________________________
Relationship to participant: _________________________
Email: _________________________
Phone: _________________________
C2. Highest level of education completed by any adult in household:
Less than high school
High school diploma or GED
Some college or associate degree
Bachelor's degree
Graduate or professional degree
Prefer not to say
C3. How would you describe your household's economic situation?
Struggling to meet basic needs
Getting by with careful budgeting
Comfortable
More than enough
Prefer not to say
C4. Is the participant eligible for free/reduced lunch at school?
Yes
No
Don't know
Not applicable
SECTION D: Permissions
D1. Photo/Video Permission:
YES — Photos/videos of my child may be used for steamHouse program documentation and improvement
NO — Please do not photograph or video my child
D2. Data Use Permission: De-identified data (no names or identifying information) may be used in aggregate reports and research about program effectiveness.
YES — I consent to de-identified data use
NO — Please do not include my child's data in aggregate reports
D3. Future Contact Permission: We may want to follow up with participants after the program ends to learn about longer-term outcomes.
YES — You may contact us for follow-up
NO — Please do not contact us after the program
Signatures:
Participant signature (if 12+): _________________________ Date: _________
Parent/Guardian signature: _________________________ Date: _________
This form will be stored securely. Only program staff and authorized researchers will have access to individual responses. All published reports will use aggregate data only.
INSTRUMENT 2: MENTOR WEEKLY FIDELITY LOG
Purpose
Track what steamHouse content was actually delivered, how sessions went, and maintain observational notes on participant engagement. Essential for understanding implementation fidelity and dosage.
Administration
When: End of each program session/meeting
How: Paper form, online survey, or app
Who completes: Mentor/coach/facilitator
Time: 3-5 minutes per session
STEAMHOUSE MENTOR WEEKLY FIDELITY LOG
Session Information
FieldResponseSite/Program_________________________Date//________Session #_______ of _______ plannedSession length_______ minutesMentor completing log_________________________
SECTION A: Attendance
A1. Number of participants present today: _______
A2. Number of participants absent: _______
A3. Any new participants this session?
No
Yes — How many? _______
SECTION B: steamHouse Content Delivered
B1. Did you use any steamHouse curriculum content this session?
Yes
No (skip to Section C)
B2. Which steamHouse elements were incorporated? (check all that apply)
Framework/Principles:
Four Principles explicitly mentioned (Agency, Respect, Reason, Reflection)
Three Levels of Consciousness (Automatic/Conscious/Purposeful)
Purpose-Paradigm-Practice model
Gold Star Kit concept
Specific Lenses (Thinking Frameworks):
Scout vs. Soldier Mindset
Ladder of Inference
Pre-Mortem
Growth Mindset discussion
Other Lens: _________________________
Specific Keys (Skills Practice):
Active Listening exercise
Checklist use
Feedback giving/receiving
Other Key: _________________________
Reflection Activities:
Individual written reflection
Group debrief/discussion
What went well / What would we change
Other reflection: _________________________
B3. Approximately how many minutes were dedicated to explicit steamHouse content? _______ minutes (estimate)
B4. How well did the steamHouse content integrate with the activity?
1 — Felt forced/awkward
2 — Somewhat integrated
3 — Mostly smooth
4 — Natural fit
5 — Perfectly integrated
SECTION C: Session Quality
C1. Overall, how would you rate this session?
1 — Poor (major problems)
2 — Below average
3 — Average
4 — Good
5 — Excellent
C2. Participant engagement level:
1 — Very low (distracted, disinterested)
2 — Low
3 — Moderate
4 — High
5 — Very high (fully engaged)
C3. Any challenges this session? (check all that apply)
None
Attendance issues
Behavioral challenges
Time constraints
Material/equipment problems
Participant conflict
Content too difficult
Content too easy
Other: _________________________
SECTION D: Notable Observations
D1. Did you observe any participant(s) demonstrating steamHouse concepts without prompting? (e.g., using framework language, helping others, showing metacognition)
No
Yes — Brief description:
D2. Any "bright spots" or particularly positive moments?
D3. Any concerns about specific participants? (Note: Use participant ID, not name)
D4. What would you do differently next time?
SECTION E: Quick Participant Ratings (Optional — for Research-Ready Package)
Rate each participant present today on the following (1-5 scale):
Participant IDEngagement (1-5)Framework Use (1-5)Peer Support (1-5)________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Rating Key:
Engagement: 1=Disengaged, 5=Fully absorbed
Framework Use: 1=None observed, 5=Spontaneous sophisticated use
Peer Support: 1=No interaction, 5=Active helping/teaching others
Log completed by: _________________________ Time: _________
INSTRUMENT 3: MONTHLY PARTICIPANT REFLECTION
Purpose
Capture evidence of metacognitive development through structured reflection prompts. Designed to show progression in reflective capacity over time without being a "test."
Administration
When: Monthly (or at natural program milestones)
How: Paper journal, online form, or audio recording option
Who completes: Participant
Time: 5-10 minutes
Age Adaptations
Ages 8-10: Simplified prompts, drawing option, mentor scribing allowed
Ages 11-13: Standard prompts
Ages 14+: Full prompts with optional extension questions
STEAMHOUSE MONTHLY REFLECTION
Participant ID: _________________________ Date: //________
Month #: _______ of program
PROMPT 1: A Decision or Problem
Think about something you had to figure out or decide this month—in this program or in your life.
What was the situation?
What did you think about before deciding or acting?
Did you notice yourself thinking in any particular way? (For example: Did you slow down? Ask questions? Consider different viewpoints? Think about what could go wrong?)
PROMPT 2: Something You Learned
Think about something you learned or got better at this month.
What was it?
How do you know you learned it? (What's your evidence?)
What helped you learn it?
PROMPT 3: Working with Others
Think about a time this month when you worked with others on your team.
What happened?
What did you do that helped the team?
What did you notice about how the team worked together?
PROMPT 4: Self-Rating (Circle one for each)
This month, how often did you...
Stop and think before reacting? Never | Rarely | Sometimes | Often | Almost Always
Notice what you were feeling? Never | Rarely | Sometimes | Often | Almost Always
Ask yourself "Is this actually true?" Never | Rarely | Sometimes | Often | Almost Always
Consider someone else's perspective? Never | Rarely | Sometimes | Often | Almost Always
Think about what could go wrong before acting? Never | Rarely | Sometimes | Often | Almost Always
PROMPT 5: Framework Check-In (Ages 11+)
Below are some thinking approaches. Check any you remember using this month:
Asked "Am I being a Scout or a Soldier?" (open vs. defensive)
Climbed down the Ladder of Inference (checked my assumptions)
Did a Pre-Mortem (imagined what could go wrong)
Remembered that my brain can change with practice
Noticed the difference between reacting and responding
Thought "What would the growth mindset version of me do?"
Used a checklist to make sure I didn't miss something
Practiced active listening (really trying to understand)
Other: _________________________
EXTENSION QUESTIONS (Ages 14+ or optional)
Looking back at your reflections from last month (if available), do you notice any changes in how you think or approach problems?
Is there a thinking framework or skill you want to work on next month? Why?
YOUNGER PARTICIPANT VERSION (Ages 8-10)
Mentor may read prompts aloud and scribe responses. Drawing is acceptable.
1. Tell me about a problem you had to solve this month. (What happened? What did you do?)
2. What's something you got better at?
3. Draw or write: A time you helped someone on your team
[SPACE FOR DRAWING]
4. Circle the face that shows how much you think before you act:
😶 (Never) | 🙂 (Sometimes) | 😊 (A lot) | 🤩 (Almost always)
Reflection completed on: //________
INSTRUMENT 4: DEVELOPMENT MARKER SELF-ASSESSMENT
Purpose
Track self-perceived progress on the 53 steamHouse Development Markers. Administered at entry and exit (minimum) to show change over time. NOT a validated instrument—designed for self-reflection and progress tracking.
Administration
When: Program entry and program exit (minimum); optionally mid-point
How: Paper form or online survey
Who completes: Participant (mentor assists for ages 8-11)
Time: 15-20 minutes
Important Notes
This is self-assessment, not objective measurement
Frame as "where you see yourself now" not "test"
No right/wrong answers
Change over time is the interest, not absolute scores
STEAMHOUSE DEVELOPMENT MARKER SELF-ASSESSMENT
Participant ID: _________________________
Date: //________
Assessment Point: [ ] Entry | [ ] Mid-Point | [ ] Exit
Instructions
For each statement below, mark how much this describes you RIGHT NOW.
Response Options:
1 = Not at all like me — This doesn't describe me
2 = A little like me — This sometimes describes me
3 = Somewhat like me — This often describes me
4 = A lot like me — This usually describes me
5 = This is me — This describes me almost all the time
There are no right or wrong answers. We want to know how you honestly see yourself today.
PART A: STARS — Character Qualities (14 items)
How I am as a person...
#StatementRating (1-5)S1I believe my brain can grow and change with practice_____S2When things are hard, I see it as a chance to learn and improve_____S3I can usually tell what emotion I'm feeling in the moment_____S4I'm good at noticing how other people are feeling_____S5I have different ways to handle my emotions depending on the situation_____S6When bad things happen, I see them as temporary and specific, not permanent and everywhere_____S7I stick with important goals even when it's hard or takes a long time_____S8I have a clear sense of why my life and what I do matters_____S9I understand what triggers my habits and what rewards keep them going_____S10I'm good at protecting my attention from distractions_____S11I'm genuinely curious about people and ideas—I really want to understand_____S12I try to make other people feel seen and understood, not to impress them_____S13I feel connected to my community and responsible for others' wellbeing_____S14Even when I disagree with someone, I can see them as a full person, not just an obstacle_____
PART B: LENSES — Thinking Frameworks (22 items)
How I think and make decisions...
#StatementRating (1-5)L1I try to find the truth rather than defend what I already believe_____L2When I react to something, I check whether my assumptions are actually true_____L3I know how to spot bad arguments and unreliable information_____L4I consider how common something is before judging how likely it is_____L5I think in terms of probabilities (likely/unlikely) rather than certainties_____L6I judge my decisions by my reasoning, not just by how they turned out_____L7Before starting something important, I imagine what could go wrong_____L8I have a clear process for making important decisions_____L9I know when "good enough" is the right answer vs. when to keep searching_____L10For big decisions, I think about how I'll feel in 10 minutes, 10 months, and 10 years_____L11I treat reversible decisions differently than irreversible ones_____L12I understand how things build up (stocks) and how they flow in/out_____L13I notice when actions create cycles that reinforce or balance themselves_____L14I know how to practice deliberately to actually improve_____L15I understand the conditions that help me get into focused "flow" states_____L16I know that some difficulty in learning actually helps me learn better_____L17I test myself on what I'm learning rather than just re-reading_____L18When conversations are difficult, I can tell if it's about facts, feelings, or identity_____L19I understand different patterns of how people connect in relationships_____L20I can recognize criticism, contempt, defensiveness, and stonewalling in interactions_____L21I notice when people reach out for connection and try to respond positively_____L22I can tell when I'm seeing someone as a problem vs. as a person_____
PART C: KEYS — Practical Skills (17 items)
What I can actually do...
#StatementRating (1-5)K1I consistently get good sleep (enough hours, regular schedule)_____K2I include physical movement in my regular routine_____K3I can tell the difference between real danger and just feeling nervous_____K4I can set boundaries with people when I need to_____K5I automatically save some money/resources before spending_____K6I help create environments where people feel safe to take risks_____K7I capture ideas and tasks somewhere reliable so I don't forget_____K8I clarify the very next action I need to take to move things forward_____K9I use checklists for important or complex tasks_____K10I focus on doing the process well rather than just getting outcomes_____K11When someone talks, I really listen to understand, not just wait to talk_____K12I can receive feedback without getting defensive_____K13I can express my needs without blaming or attacking_____K14I ask good questions that help understanding_____K15After a conflict, I try to repair the relationship_____K16I'm willing to show vulnerability first to build trust_____K17When negotiating, I try to understand what people really need, not just what they're asking for_____
PART D: Overall Reflection
D1. Looking at all your ratings, which THREE items would you most like to develop?
Item #: _____ Why: _________________________________________________
Item #: _____ Why: _________________________________________________
Item #: _____ Why: _________________________________________________
D2. Which THREE items are you proudest of?
Item #: _____
Item #: _____
Item #: _____
D3. Is there anything about yourself that these questions didn't capture?
SCORING SUMMARY (For Mentor/Researcher Use)
DomainItemsSumAverageStars (Character)S1-S14__________Lenses (Thinking)L1-L22__________Keys (Skills)K1-K17__________TOTALAll 53__________
Change from Entry to Exit: (Exit average - Entry average)
DomainEntry AvgExit AvgChangeStars_______________Lenses_______________Keys_______________TOTAL_______________
Assessment completed on: //________
INSTRUMENT 5: PILOT PARTNER DATA AGREEMENT
Purpose
Establish clear expectations between steamHouse and pilot partner organizations about what data will be collected, how it will be used, and what each party provides.
Use
Signed before pilot begins. Adapted to specific partner and data package selected.
STEAMHOUSE PILOT PARTNERSHIP DATA AGREEMENT
PARTIES
steamHouse ("Provider") Contact: _________________________ Email: _________________________ Address: _________________________
Partner Organization ("Partner") Organization Name: _________________________ Contact Person: _________________________ Title: _________________________ Email: _________________________ Address: _________________________
Effective Date: //________
1. PURPOSE
This Agreement establishes the terms for data collection during the pilot implementation of steamHouse curriculum within Partner's program(s). The purpose of data collection is to:
Support continuous improvement of steamHouse curriculum and implementation methods
Document program reach and participation
Track implementation fidelity
Gather preliminary evidence of program effects
Prepare for future formal research evaluation
2. PROGRAM DETAILS
Activity/Program Type: _________________________
Participant Age Range: _______ to _______ years
Expected Number of Participants: _______
Program Duration: //________ to //________
Number of Sessions: _______ (estimated)
Session Frequency: _________________________
3. DATA COLLECTION PACKAGE SELECTED
[ ] Minimal Package
Participant Intake Form (entry)
Attendance tracking (ongoing)
Exit Survey (end)
Mentor session count
[ ] Standard Package
All Minimal Package items, plus:
Monthly Participant Reflection
Development Marker Self-Assessment (entry + exit)
Mentor Weekly Fidelity Log
[ ] Research-Ready Package
All Standard Package items, plus:
Monthly Mentor Observation Ratings
Parent/Guardian Survey (optional)
Comparison data (if available)
4. PARTNER RESPONSIBILITIES
Partner agrees to:
4.1 Participant Recruitment & Consent
Distribute and collect Participant Intake Forms including consent language
Ensure parent/guardian consent is obtained for all participants under 18
Respect participant/family choice to decline data collection while still participating in program
4.2 Data Collection
Administer selected instruments according to the schedule provided
Ensure participants have adequate time and environment to complete instruments thoughtfully
Submit completed instruments to steamHouse within [14] days of collection
4.3 Implementation
Complete Mentor Weekly Fidelity Logs for each program session
Attend steamHouse mentor orientation/training (provided by steamHouse)
Communicate any implementation challenges or adaptations to steamHouse
4.4 Confidentiality
Store any paper instruments securely
Limit access to participant information to authorized personnel only
Report any data breaches or confidentiality concerns immediately
5. STEAMHOUSE RESPONSIBILITIES
steamHouse agrees to:
5.1 Materials & Support
Provide all data collection instruments in requested format (paper/digital)
Provide mentor training on curriculum implementation and data collection
Respond to questions and provide technical support throughout pilot
5.2 Data Management
Assign unique Participant IDs to protect confidentiality
Store all data securely using [describe system]
Remove identifying information before any analysis or reporting
5.3 Reporting
Provide Partner with summary report of their program's data within [60] days of pilot completion
Share aggregate findings that may inform Partner's program improvement
Not publish or publicly present Partner-specific data without written permission
5.4 Confidentiality
Maintain confidentiality of all participant information
Use data only for purposes described in this Agreement
Not share individual-level data with third parties without consent
6. DATA OWNERSHIP AND USE
6.1 Ownership
Partner owns data about their participants
steamHouse has license to use de-identified data for program improvement and research
6.2 Permitted Uses by steamHouse
Internal program improvement and curriculum development
Aggregate reporting (no individual or Partner identification)
Academic publication (de-identified, aggregate data only)
Grant applications (aggregate outcomes only)
Future formal research (with additional IRB approval if required)
6.3 Prohibited Uses
Individual participant data will never be sold or shared with third parties for marketing
Partner-specific results will not be published without Partner's written consent
Data will not be used to evaluate individual mentor performance for employment purposes
7. PARTICIPANT RIGHTS
7.1 Voluntary Participation Participation in data collection is voluntary. Participants may:
Decline to complete any instrument
Skip any question
Withdraw from data collection at any time
Continue participating in the program regardless of data collection participation
7.2 Confidentiality
Participant names will not be used in any reports or publications
Participant IDs will be used for data tracking
Only aggregate results will be shared publicly
7.3 Access
Parents/guardians may request to see their child's individual responses
Participants 18+ may request their own data
8. DURATION AND TERMINATION
8.1 Term This Agreement is effective from the date signed until [90] days after the pilot program ends.
8.2 Termination Either party may terminate this Agreement with [30] days written notice. Upon termination:
Partner will submit any outstanding data collection instruments
steamHouse will provide summary report of data collected to date
steamHouse may retain de-identified data for program improvement
9. SIGNATURES
For steamHouse:
Signature: _________________________ Date: //________
Print Name: _________________________
Title: _________________________
For Partner Organization:
Signature: _________________________ Date: //________
Print Name: _________________________
Title: _________________________
APPENDIX A: Data Collection Schedule
InstrumentWhenAdministered BySubmitted ToParticipant Intake FormProgram entryPartnersteamHouseMentor Weekly Fidelity LogAfter each sessionMentorsteamHouseMonthly Participant ReflectionMonthlyParticipantsteamHouse via PartnerDevelopment Marker Self-AssessmentEntry + ExitParticipantsteamHouse via PartnerExit SurveyProgram endPartnersteamHouse
APPENDIX B: Contact Information
For questions about data collection instruments: steamHouse Contact: _________________________ Email: _________________________ Phone: _________________________
For technical issues with online forms: Email: _________________________
For urgent confidentiality concerns: Contact: _________________________ Phone: _________________________
Agreement Version 1.0 | December 2025
APPENDIX: ADDITIONAL INSTRUMENTS
Exit Survey (Short Form)
For use with Minimal Package. Quick assessment at program end.
STEAMHOUSE EXIT SURVEY
Participant ID: _________________________ Date: //________
1. Overall, how would you rate your experience in this program?
Poor
Fair
Good
Very Good
Excellent
2. How much did you learn in this program?
Nothing
A little
Some
A lot
A huge amount
3. Would you recommend this program to a friend?
Definitely not
Probably not
Maybe
Probably yes
Definitely yes
4. What was the best part of the program?
5. What could be improved?
6. Did this program change how you think about things?
No, not at all
A little
Somewhat
Yes, definitely
If yes, how? _________________________________________________________________
7. Anything else you want to tell us?
Parent/Guardian Survey (Optional)
For Research-Ready Package. Gathers outside perspective on participant development.
STEAMHOUSE PARENT/GUARDIAN SURVEY
Participant ID: _________________________ Date: //________
Relationship to participant: _________________________
1. Have you noticed any changes in your child since they started this program?
No changes
Minor positive changes
Significant positive changes
Minor negative changes
Significant negative changes
2. If you noticed changes, please describe:
3. Has your child talked about things they learned in the program?
Never
Rarely
Sometimes
Often
Very often
4. Have you noticed your child using any new approaches to thinking or problem-solving?
No
Maybe/not sure
Yes
If yes, please describe: _________________________________________________________________
5. How satisfied are you with this program overall?
Very dissatisfied
Dissatisfied
Neutral
Satisfied
Very satisfied
6. Would you enroll your child in similar programs in the future?
Definitely not
Probably not
Maybe
Probably yes
Definitely yes
7. Any other comments or feedback?