Classroom Activity Dashboard

15-second coded observation · classroom video

This dashboard visualizes a 15-second classroom video, coded second-by-second by Gemini 3.1 Pro into eight observable student behaviors. The video comes from a paper which created and annotated a classroom video dataset for group engagement recognition and demonstrated how existing computer vision models perform on it.

Read the Paper Videos from Paper

Activities Observed, Overall

Total seconds across all students · s combined

Activities Identified Each Second

Hover a column for the student-level breakdown

Activities by Student

Hover a bar for that student's activity seconds

What This Clip Shows

Based on the model's per-second, per-student output above

Open Raw JSON Output ↗ Download JSON

Per-Second Timeline

One row per student per second · use the dropdown or the search box in each column to filter

Open Raw JSON Output ↗ Download JSON

Events

One row per detected event · use the dropdown or the search box in each column to filter

Activity Summaries

Totals across all events, for every activity type in the taxonomy

Student Summaries

One row per student per activity · use the dropdown or the search box in each column to filter

Validated Events

One row per event · use the dropdown or the search box in each column to filter

Analysis Pipeline - Google AI Studio

Model Configuration

Model: Gemini 3.1 Pro
Temperature: 1
Top P: 0.95
Max Output: 65,536 tokens
Grounding: Off
Safety: All Off
Thinking Level: High
Media Resolution: Default
Structured Outputs: On

System Instructions

You are an expert Educational Data Scientist specializing in classroom behavior analysis. Your task is to analyze the provided classroom video and produce objective, structured observations describing visible student behaviors. Use an event-driven approach. Do not produce a second-by-second timeline. DEFINITION OF AN EVENT An event is a continuous period during which one or more students visibly perform the same activity. An event begins at the first second when the activity is clearly visible. An event ends at the last second when the activity remains clearly visible. Treat an activity as one continuous event when: - the same activity continues across consecutive seconds; - the number of participating students changes while the activity continues; - a student is briefly obscured but appears to continue the same activity; - the activity is interrupted for one second or less and then clearly resumes. Create a new event when: - the activity clearly stops for more than one second before restarting; - a different group of students independently begins the same activity; - the observable activity changes to a different activity; - there is insufficient evidence that two observations belong to the same continuous event. When multiple students perform the same activity at the same time as part of the same classroom occurrence, record one event and report how many students participated. When separate groups independently perform the same activity at the same time, record separate events when the groups can be reliably distinguished. STUDENT IDENTIFICATION AND TRACKING If possible, maintain consistent anonymous identifiers for students throughout the video, such as Student_1, Student_2, and Student_3. Never identify students by name or infer personal characteristics. If reliable tracking is not possible because students become occluded, leave the frame, change seats, or cannot be distinguished, use temporary anonymous identifiers when necessary. Report whether reliable student tracking was achieved in the metadata. Do not force consistent identifiers when the evidence does not support reliable tracking. ACTIVITY CATEGORIES For every qualifying event, assign one of the following observable activities: - Listening_to_Lecture - Note_Taking - One_to_One_Discussion - Group_Discussion - Using_Phone - Looking_Away - Handling_Materials - Hand_Raised - Answering_Question - Standing - Away_From_Desk - Smiling - Other - Unknown Use only these activity categories. A student may participate in multiple simultaneous events when multiple activities are directly observable. For example, a student may be Standing while also Answering_Question. Record overlapping events separately when they represent different activity categories. Only report behaviors that are directly visible or clearly audible in the video. Do not infer: - emotions; - engagement; - attention; - understanding; - motivation; - intentions; - thoughts; - learning; - relationships between students. Smiling may be reported only when a visible smile is directly observable. Do not interpret smiling as happiness, enjoyment, or engagement. Listening_to_Lecture may be reported only when the student is visibly oriented toward an identifiable speaker, presentation, demonstration, board, or shared instructional focus. Do not infer listening from stillness alone. Looking_Away may be reported only when the student visibly turns away from the apparent instructional or group focus. Do not interpret Looking_Away as disengagement. Use Unknown when an activity is visible but cannot be classified confidently. Use Other only when an observable activity is clear but does not fit another listed category. Briefly describe the activity in the event notes. EVENT TIMING Report event times in both: - whole seconds from the beginning of the video; and - MM timestamp format. Round timestamps to the nearest whole second. For every event, report: - a sequential event identifier; - activity; - start second; - end second; - start timestamp; - end timestamp; - duration in seconds; - number of students participating when the event begins; - minimum number of participating students during the event; - maximum number of participating students during the event; - participation pattern; - anonymous student identifiers, when reliable; - confidence; - a brief factual description of the visible evidence; - any event-specific limitations. Calculate duration using: duration_seconds = end_second - start_second + 1 STUDENT COUNTS Count only students who visibly participate in the event. Do not count a student merely because the student is nearby or visible in the frame. Report: - students_at_start: number participating when the event begins; - minimum_students: lowest number participating during the event; - maximum_students: highest number participating during the event; - participation_pattern: stable, increased, decreased, or varied. If the exact count is uncertain, report the most defensible count, lower the confidence level, and explain the uncertainty in event_limitations. VIDEO-LEVEL SUMMARIES After completing the event analysis, calculate a summary for every activity category. For each activity, report: - total number of distinct events; - total duration of all events; - average event duration; - minimum event duration; - maximum event duration; - largest number of students participating in a single event; - total student-event participations. Total student-event participations means the sum of the maximum number of participating students across all events of that activity. This is not a count of unique students and must not be labeled as one. Because events may overlap: - do not interpret summed event durations as the total non-overlapping length of the video; - do not combine overlapping activities unless they belong to the same activity category and meet the continuous-event rules. STUDENT SUMMARIES Create student-level summaries only when student tracking is sufficiently reliable. For each reliably tracked student and activity, report: - number of distinct events in which the student participated; - total duration of those events. Do not include a student-level summary for temporary or unreliable identifiers unless the identifier can be followed consistently for a meaningful portion of the video. VISIBLE-STUDENT SAMPLING Estimate the average number of students visible by sampling the video at approximately one-second intervals internally. Do not return the one-second sampling timeline. Return only the resulting average in the metadata. QUALITY CONTROL Before returning the final output: - Review the complete video. - Confirm that the output contains events rather than a second-by-second timeline. - Confirm that continuous activities have not been divided into duplicate events. - Confirm that activities separated by more than one second have not been incorrectly combined. - Confirm that separate groups have not been combined when they can be reliably distinguished. - Confirm that every start time is less than or equal to its end time. - Confirm that every duration equals end_second minus start_second plus one. - Confirm that all timestamps fall within the video duration. - Confirm that each activity summary matches the corresponding event records. - Confirm that student summaries are included only when tracking is sufficiently reliable. - Confirm that only directly observable behaviors are reported. - Confirm that every defined activity appears in activity_summaries, including activities with zero detected events. Return only data matching the provided structured output schema.

Structured Output Schema (JSON Schema)

{
  "type": "object",
  "properties": {
    "events": {
      "type": "array",
      "description": "Distinct observable activity events ordered by start_second and then by event_id. This must not be a second-by-second timeline.",
      "items": {
        "type": "object",
        "properties": {
          "event_id": { "type": "integer", "description": "Sequential event number beginning with 1." },
          "activity": {
            "type": "string",
            "enum": ["Listening_to_Lecture","Note_Taking","One_to_One_Discussion","Group_Discussion","Using_Phone","Looking_Away","Handling_Materials","Hand_Raised","Answering_Question","Standing","Away_From_Desk","Smiling","Other","Unknown"]
          },
          "start_second": { "type": "integer", "description": "First whole second at which the event is clearly observable." },
          "end_second": { "type": "integer", "description": "Final whole second at which the event remains clearly observable." },
          "start_timestamp": { "type": "string", "description": "Event start time in MM format." },
          "end_timestamp": { "type": "string", "description": "Event end time in MM format." },
          "duration_seconds": { "type": "integer", "description": "Inclusive duration calculated as end_second minus start_second plus 1." },
          "students_at_start": { "type": "integer", "description": "Number of students visibly participating when the event begins." },
          "minimum_students": { "type": "integer", "description": "Smallest number of visibly participating students during the event." },
          "maximum_students": { "type": "integer", "description": "Largest number of visibly participating students during the event." },
          "participation_pattern": {
            "type": "string",
            "enum": ["stable","increased","decreased","varied"],
            "description": "How the number of participating students changed during the event."
          },
          "student_ids": {
            "type": "array",
            "description": "Anonymous identifiers for students who participated in the event when reliable tracking was possible. Return an empty array when reliable identification was not possible.",
            "items": { "type": "string" }
          },
          "confidence": {
            "type": "string",
            "enum": ["high","medium","low"],
            "description": "Confidence in the activity classification, timestamps, and student count."
          },
          "evidence": { "type": "string", "description": "Brief, objective description of the visible or audible evidence supporting the event classification." },
          "event_limitations": { "type": "string", "description": "Event-specific uncertainty caused by occlusion, image quality, camera angle, uncertain counts, or similar limitations. Use an empty string when no specific limitation applies." }
        },
        "required": ["event_id","activity","start_second","end_second","start_timestamp","end_timestamp","duration_seconds","students_at_start","minimum_students","maximum_students","participation_pattern","student_ids","confidence","evidence","event_limitations"]
      }
    },
    "activity_summaries": {
      "type": "array",
      "description": "One summary record for every defined activity, including activities with zero detected events.",
      "items": {
        "type": "object",
        "properties": {
          "activity": {
            "type": "string",
            "enum": ["Listening_to_Lecture","Note_Taking","One_to_One_Discussion","Group_Discussion","Using_Phone","Looking_Away","Handling_Materials","Hand_Raised","Answering_Question","Standing","Away_From_Desk","Smiling","Other","Unknown"]
          },
          "event_count": { "type": "integer", "description": "Number of distinct events detected for this activity." },
          "total_duration_seconds": { "type": "integer", "description": "Sum of duration_seconds across all events of this activity. Overlapping events may cause totals across activities to exceed the video duration." },
          "average_event_duration_seconds": { "type": "number", "description": "Average duration of events of this activity. Return 0 when event_count is 0." },
          "minimum_event_duration_seconds": { "type": "integer", "description": "Duration of the shortest event of this activity. Return 0 when event_count is 0." },
          "maximum_event_duration_seconds": { "type": "integer", "description": "Duration of the longest event of this activity. Return 0 when event_count is 0." },
          "largest_student_count": { "type": "integer", "description": "Largest maximum_students value among events of this activity. Return 0 when event_count is 0." },
          "total_student_event_participations": { "type": "integer", "description": "Sum of maximum_students across events of this activity. This is not a count of unique students." }
        },
        "required": ["activity","event_count","total_duration_seconds","average_event_duration_seconds","minimum_event_duration_seconds","maximum_event_duration_seconds","largest_student_count","total_student_event_participations"]
      }
    },
    "student_summaries": {
      "type": "array",
      "description": "Student-level summaries included only for students who could be tracked with sufficient reliability.",
      "items": {
        "type": "object",
        "properties": {
          "id": { "type": "string", "description": "Anonymous student identifier." },
          "activities": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "activity": {
                  "type": "string",
                  "enum": ["Listening_to_Lecture","Note_Taking","One_to_One_Discussion","Group_Discussion","Using_Phone","Looking_Away","Handling_Materials","Hand_Raised","Answering_Question","Standing","Away_From_Desk","Smiling","Other","Unknown"]
                },
                "event_count": { "type": "integer", "description": "Number of distinct events of this activity in which the student participated." },
                "total_event_duration_seconds": { "type": "integer", "description": "Sum of the full durations of events of this activity in which the student participated." }
              },
              "required": ["activity","event_count","total_event_duration_seconds"]
            }
          }
        },
        "required": ["id","activities"]
      }
    },
    "metadata": {
      "type": "object",
      "properties": {
        "video_duration_seconds": { "type": "integer", "description": "Approximate total duration of the analyzed video in whole seconds." },
        "average_students_visible": { "type": "number", "description": "Average number of visible students estimated through internal sampling at approximately one-second intervals." },
        "maximum_students_visible": { "type": "integer", "description": "Largest number of students visible at one time." },
        "reliable_tracking": { "type": "boolean", "description": "Whether students could be assigned consistent anonymous identifiers throughout most of the video." },
        "tracking_notes": { "type": "string", "description": "Explanation of student-tracking reliability, including use of temporary identifiers." },
        "event_gap_rule_seconds": { "type": "integer", "description": "Maximum interruption that was merged into the same event. This should normally equal 1." },
        "limitations": { "type": "array", "description": "Video-level limitations affecting interpretation, counting, timing, or tracking.", "items": { "type": "string" } }
      },
      "required": ["video_duration_seconds","average_students_visible","maximum_students_visible","reliable_tracking","tracking_notes","event_gap_rule_seconds","limitations"]
    }
  },
  "required": ["events","activity_summaries","student_summaries","metadata"]
}

Read the Paper View the Data

Validated Per-Second Timeline

One row per student per second · use the dropdown or the search box in each column to filter

Analysis Pipeline - Google AI Studio

Model Configuration

Model: Gemini 3.1 Pro
Temperature: 1
Top P: 0.95
Max Output: 65,536 tokens
Grounding: Off
Safety: All Off
Thinking Level: High
Media Resolution: Default
Structured Outputs: On

System Instructions

You are an expert Educational Data Scientist specializing in classroom behavior analysis. Your task is to analyze the provided classroom video and produce objective, structured observations describing the visible behaviors of students. Analyze the video at approximately one observation per second. For each second: - Count the number of visible students. - If possible, maintain a consistent anonymous identifier for each student throughout the video (Student_1, Student_2, Student_3, etc.). - If reliable tracking is not possible because students become occluded or leave the frame, continue assigning temporary identifiers and indicate this in the final metadata. For every visible student, assign one or more of the following observable activities: - Listening_to_Lecture - Note_Taking - One_to_One_Discussion - Group_Discussion - Using_Phone - Looking_Away - Handling_Materials - Hand_Raised - Answering_Question - Standing - Away_From_Desk - Smiling - Other - Unknown A student may be assigned multiple activities simultaneously. Only report behaviors that are directly visible in the video. Do not infer emotions, engagement, understanding, intentions, or thoughts. If an activity cannot be determined confidently, assign "Unknown" rather than guessing. After completing the video analysis, calculate: - Total seconds spent in each activity across all students. - If student tracking was reliable, total seconds spent in each activity for each student. - Average number of students visible. - Whether reliable student tracking was achieved. - Any limitations affecting the analysis (occlusion, poor visibility, students leaving the frame, etc.). Return only data matching the provided structured output schema.

Structured Output Schema (JSON Schema)

{
  "type": "object",
  "properties": {
    "timeline": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "second": { "type": "integer" },
          "students_present": { "type": "integer" },
          "students": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "id": { "type": "string" },
                "activities": {
                  "type": "array",
                  "items": {
                    "type": "string",
                    "enum": ["Listening_to_Lecture","Note_Taking","One_to_One_Discussion","Group_Discussion","Using_Phone","Looking_Away","Handling_Materials","Hand_Raised","Answering_Question","Standing","Away_From_Desk","Smiling","Other","Unknown"]
                  }
                }
              },
              "required": ["id","activities"]
            }
          }
        },
        "required": ["second","students_present","students"]
      }
    },
    "activity_totals": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "activity": {
            "type": "string",
            "enum": ["Listening_to_Lecture","Note_Taking","One_to_One_Discussion","Group_Discussion","Using_Phone","Looking_Away","Handling_Materials","Hand_Raised","Answering_Question","Standing","Away_From_Desk","Smiling","Other","Unknown"]
          },
          "seconds": { "type": "integer" }
        },
        "required": ["activity","seconds"]
      }
    },
    "student_summaries": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": { "type": "string" },
          "activities": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "activity": {
                  "type": "string",
                  "enum": ["Listening_to_Lecture","Note_Taking","One_to_One_Discussion","Group_Discussion","Using_Phone","Looking_Away","Handling_Materials","Hand_Raised","Answering_Question","Standing","Away_From_Desk","Smiling","Other","Unknown"]
                },
                "seconds": { "type": "integer" }
              },
              "required": ["activity","seconds"]
            }
          }
        },
        "required": ["id","activities"]
      }
    },
    "metadata": {
      "type": "object",
      "properties": {
        "average_students_visible": { "type": "number" },
        "reliable_tracking": { "type": "boolean" },
        "limitations": { "type": "string" }
      },
      "required": ["average_students_visible","reliable_tracking","limitations"]
    }
  },
  "required": ["timeline","activity_totals","student_summaries","metadata"]
}

Read the Paper View the Data