Computer Vision for Hockey Video Curation

Computer vision-based models are being actively investigated for tasks such as ball and player tracking. These insights are useful for both coaches and players to improve performance. Applying computer vision-based solutions for hockey video analysis is challenging because of the small size of the puck, fast and non-smooth movement of the players, and frequent occlusions. In this thesis, I present my research work on computer vision for hockey video curation. I discuss three problems: 1) automatic sports videography, 2) play segmentation of hockey videos and 3) automatic homography estimation.

When recording broadcast hockey videos, professional cameramen move a PTZ camera to follow the play. Professional videography is expensive for amateur games and this motivates the development of a low-cost solution for automatic hockey videography. We used a novel method for accurate ground truth of the puck location from wide-field video. We trained a novel deep network regressor to estimate the puck location on each frame. Centered around the predicted puck location, we dynamically cropped the wide-field video to generate a zoomed-in video following the play.

The automatic videography system delivers continuous video over the entire game. Typical hockey games that feature 40-60 minutes of active play are played over 60-110 minutes with breaks in play due to warm-ups and fouls. We propose a novel solution for automatically identifying periods of play and no-play, and generate a temporally compressed video that is easier to watch. We combine visual cues from the output of a deep network classifier with auditory cues from the referee's whistle using a hidden Markov model (HMM).

Since the PTZ parameters of the camera are constantly varying when recording broadcast hockey videos, the homography changes every frame. Knowing this homography allows for the projection of graphics onto the ice surface. We estimate the homography by exploiting the consistency of colours used for markings in ice hockey. We model the colours as a multi-variate Gaussian and then use a two-step approach to search for the homography that aligns the colours of the template image to that of a test image.