Introduction

This project originates from automated game evaluation. A fisheye camera is used to capture a room where people are playing a newly designed social game. In the gram, two teams compete with each other and the team dancing in higher sync win the game.

In the previous work, people are detected and marked with a bounding box as in Fig. 1. In the work, we track and clustering people into groups. People in each group should have similar behavior and group is the minimal study unit in the following game evaluation.

Fig. 1 Detected people are marked with a boundary box.

Tracking

The basic idea of tracking is searching for the most similar boundary box in the coming frame. However, in fisheye video, as people move, the bounding boxes undergo significant deformation and it needs to be taken into consideration when comparing box contents. We handle it by reconstructing the real scene from the fisheye video based on fisheye imaging model and using persisting physical constants as the clues to predict the box deformation. Of course, it is really possible to reconstruct the real scene from a single fisheye camera. Strong assumption about the real scene is used which is enough for the tracking task.

Fig. 2 People tracking showcase 1.
Fig. 3 People tracking showcase 2.
Fig. 4 People tracking showcase 3.

Clustering

After tracking, the trajectory of each people is obtained. Meanshift is then used for clustering, using people location and motion energy spectrum as features.

Fig. 5 People clustering showcase 1.
Fig. 6 People clustering showcase 2.
Back to Yuan's homepage