What it finds on the four sample clips

Part B is not scored here: evaluate.py returns null when the labels contain no accident, and none of the four sample clips has one.

Illegal U-turn scores 0.00 because we do not submit it. Our U-turn rule finds them with a mean F1 of 0.27 on these labels (0.47, 0.24, 0.12 at IoU 0.3, 0.5 and 0.7), and the annotated videos show them in purple. We leave the class out because nothing in view says a U-turn is prohibited here, so we cannot tell whether the organisers' annotators count them as illegal.

Codes: RED red light, SV stopped vehicle, JAY jaywalking, FTY failure to yield, STOP stop line, CONG congestion, UT illegal U-turn, IT illegal turn; None is no label. It checks overlap in time only, so a labelled U-turn that happens while someone jaywalks counts as jaywalking. It uses IoU 0.3 and is not one-to-one like evaluate.py, so the diagonal is higher than the TP at 0.5 in the table above. U-turns (rule off) and illegal turns (no rule) are in our labels but not in our submission, so theirs are all missed or overlap other events.

• Failure to yieldC3896 at 0:00.0north crossing, NB half
• Illegal U-turnC3896 at 0:34.9U-turn round the median nose
• Illegal U-turnC3896 at 0:37.1U-turn round the median nose
• CongestionC3896 at 0:57.5max 13 standing
• Stopped vehicleC3896 at 0:59.7NB lane
• Stopped vehicleC3896 at 0:58.6NB lane
• Stop lineC3896 at 1:06.1near the stop line
• Red lightC3896 at 1:18.9red for 12.8s
• JaywalkingC3896 at 1:48.9near the NB carriageway
• Stopped vehicleC3896 at 0:00.0NB lane
• Failure to yieldC3896 at 2:35.8north crossing, NB half
• Failure to yieldC3896 at 2:37.1north crossing, NB half
• JaywalkingC3896 at 2:43.1near the north crossing
• JaywalkingC3896 at 3:59.6near the north crossing
• CongestionC3896 at 4:34.5max 7 standing
• CongestionC3896 at 4:47.0max 13 standing
• Stop lineC3896 at 4:51.3near the stop line
• Stop lineC3897 at 0:00.0near the stop line
• JaywalkingC3897 at 0:05.8near the north crossing
• Illegal U-turnC3897 at 0:08.2U-turn round the median nose
• JaywalkingC3897 at 0:39.0near the stop line
• JaywalkingC3897 at 0:40.8near the stop line
• Illegal U-turnC3897 at 0:36.0U-turn round the median nose
• Failure to yieldC3897 at 0:55.4north crossing, SB half
• Stop lineC3897 at 0:58.8near the stop line
• Stop lineC3897 at 0:58.8near the stop line
• Failure to yieldC3897 at 1:20.0north crossing, NB half
• Illegal U-turnC3897 at 2:00.3U-turn round the median nose
• Failure to yieldC3897 at 2:36.6north crossing, NB half
• Stopped vehicleC3897 at 3:29.8junction box
• Stopped vehicleC3897 at 2:29.5NB lane
• Stopped vehicleC3897 at 3:38.5NB lane
• Failure to yieldC3902 at 0:24.1west crossing
• Illegal U-turnC3902 at 0:01.3U-turn round the median nose
• Stop lineC3902 at 1:18.9near the stop line
• JaywalkingC3902 at 1:20.4near the junction box
• Failure to yieldC3902 at 1:32.7north crossing, NB half
• Illegal U-turnC3902 at 1:23.6U-turn round the median nose
• Stop lineC3902 at 1:40.0near the stop line
• Stop lineC3902 at 1:48.2near the stop line
• Stopped vehicleC3902 at 0:00.6NB lane
• Stopped vehicleC3902 at 0:00.0NB lane
• JaywalkingC3902 at 2:39.0near the NB carriageway
• Stopped vehicleC3902 at 3:50.2NB lane
• Failure to yieldC3902 at 4:44.6west crossing
• JaywalkingC3905 at 0:11.0near the north crossing
• Failure to yieldC3905 at 0:13.4north crossing, NB half
• Stopped vehicleC3905 at 0:00.0NB lane
• Stopped vehicleC3905 at 0:30.4NB lane
• Failure to yieldC3905 at 1:20.6north crossing, NB half
• Illegal U-turnC3905 at 1:20.9U-turn round the median nose
• Stop lineC3905 at 1:15.5near the stop line
• Stop lineC3905 at 1:17.4near the stop line
• Stop lineC3905 at 1:18.9near the stop line
• Stopped vehicleC3905 at 1:19.9junction box
• JaywalkingC3905 at 1:44.0near the west crossing
• JaywalkingC3905 at 1:46.7near the west crossing
• Failure to yieldC3905 at 1:58.6north crossing, SB half

• false positiveJaywalking, C3897 at 2:51.3

  A delivery rider on a moped slows down and puts a foot on the road to let a car pass. The detector loses the moped for those frames, and at that moment the rider is too slow for our speed test to flag him as a rider, so he counts as a pedestrian standing in the junction box.

  Open at 2:51.3
• false positiveFailure to yield, C3905 at 1:08.8

  Two men wait at the kerb end of the zebra for their walk phase while a car with a green light drives past them. One of them stands on the end of the stripes and, for 0.7 s, passes both our kerb margin and our walking test, so the rule counts him as out on the crossing.

  Open at 1:08.8
• disputedStop line, C3897 at 0:58.8

  Cars in the median lane crossed the line on green, then waited about 20 s behind a car turning at the median nose. When the light went red they were still past the line, and the rule called it a stop-line violation. Our labels say it is not one; the task's definition could be read either way.

  Open at 0:58.8
• missedRed light, C3905 at 0:33.6

  A scooter creeps over the stop line about 0.9 s before green, while the red and amber lamps are lit together. Our lamp reader calls red plus amber 'yellow', and the rule also ignores crossings less than 1.5 s before green, so it never counts this one.

  Open at 0:33.6
• missedCongestion, C3896 at 2:06.5

  The southbound junction box fills up and jams through a whole green, but the jam crawls at 1 to 2 m/s instead of standing still. The rule only counts vehicles that stand, so it never sees 8 of them on green.

  Open at 2:06.5
• missedJaywalking, C3897 at 4:13.0

  A man walks up the road beside the west zebra, coming in at the bottom edge of the frame. While his feet are below the frame his foot point sits on the image edge, outside the area the rule checks, so it sees only 0.6 s of him off the crossing and needs 1 s.

  Open at 4:13.0

Class columns: RED red light, SV stopped vehicle, JAY jaywalking, FTY failure to yield, STOP stop line, CONG congestion. A dash means the class is in neither the labels nor the predictions of those clips.