Ballin – Mobi Projekt WS2020/21

Development of a sketch-based query paradigm for set play retrieval


Marc Reischmann und Mareike Müller

Project Owner: Holger Geschwindner
Project Manager: Aboubakr El Hacen Benabbas & Simon Steuer, Faculty of Information Systems and Applied Computer Sciences 
External consultants: Brandon & Nicholas Tischler

Pelle Cass, Atlanta Hawks II from ‚Crowded Field Series‘. © Pelle Cass.


  • constantly growing sports analytics market.
  • analysts spend a lot of time analyzing video material.
  • possibility to automate classification and the process to find similar plays would save lots of time and money.
  • opens new capabilities to scout opponents

Layout of the SportsVu cameras in a NBA arena in Seidl et al. (2019)


  • spatio-temporal tracking data: coordinates of all players and the ball with 25 Hz
  • standardized Information about every play-action

      Excerpt from the playbook

      Specific plays choreographed by a team are documented in the playbook of a team elaborated by the respective coach. A typical play diagram is depicted in the following. Offensive players are represented by numbers corresponding to the basic five player positions. The player who starts out with ball possession is represented by a circle with the corresponding number.

      Problem Statement

      1) Query Issuing


      Chalkboarding: sketch-based similarity query

      Sha et al (2016) proposed chalkboarding as a query paradigm for sports play retrieval. Plays can be searched by their shape. The user can draw a requested pattern (i.e. trajectories of players involved in the searched play) which serves as query format to find similar plays

      2) Retrieval

      In time series databases, in the course of a similarity query the stored trajectory and the query trajectory are compared in terms of their similarity (Yanagisawa et al., 2003)

      Similarity query for trajectories in (Yanagisawa et al. 2003, p. 8)

      Is a sketch-based similarity query suitable for set play retrieval?


      • playbook records: shape of the trajectories as key characteristic

      Our Issued Query

      • searched for fastbreaks
      • occurrence of fastbreaks has been hand recorded
      • records used for validation of our approach
      • fastbreak (Conte et al. 2017): fastest and most efficient way to make the transition from defense to offense. The fast break is most of the time the first option of every offense team due to its efficiency, which is the result of mainly two aspects:
        1. the possibility to outnumber the defensive team
        2. not allowing an effective organization of the defensive team
      • issued a query for Loyola Marymount Break (LMU) developed by  Paul Westhead (s. Kelbick 2021)
      • drawn trajectories (s. sketched query)


      Schematic drawing of the Loyola Marymount Break (LMU) developed by
      Paul Westhead (s. Kelbick 2021)

      Sketched Query


      Our retrieval system


      1. Dynamic Time Warping distance
      2. Munkres Algorithm (linear assignment problem, player alignment) (Clapper & Varoquaux, 2008)

      Two time series not aligned in the time axis illustration of linear alignment by
      Euclidean distance at the top and non-linear alignment by DTW distance at the bottom ( Keogh and Ratanamahatana 2005, p. 359)

      Database:  Golden State Warriors vs. Dallas Mavericks

      • covers the game on the 9th of February 2013 during NBA Regular Season 2012, Golden State Warriors vs. Dallas Mavericks

      • 80 Dallas Mavericks attacks, 18 labeled as fastbreakes


      Output of our tool is a list of all plays with their distances to the drawn play. The plays are ranked in decreasing order with respect to their distances.

      To evaluate the quality of our ranking, we classified the first highest ranked 18 plays as predicted fastbreaks.


      Retrieved plays (excerpt)

      True Positive (distance 24.220):

      False Negative (distance 39.214):

      False Positive (distance 37.238):

      True Negative (distance 142.046):

      Confusion Matrix

      The matrix shows that 16 of the 18 classified plays are truly labeled as fast breaks.
      Accuracy = 0.95.
      Validation on a different data set had a worse performance with an accuracy of 0.87 (Portland Trail Blazers vs. Dallas Mavericks)


      • able to retrieve plays similar to the LMU break
      • satisfactory accuracy rate and almost interactive speed
      • promising approach for simple structured plays


      • small data set
      • limited variety of search queries:
        1. very simple example
        2. no labeled data for other plays

      For more complicated plays a shape based, pattern recognition approach may not be sufficient. For more accurate and precise retrieval more information needs to be taken into account.


      • since playbook leaves some question unanswered:
        1. timing of the players for their set movement
        2. order of the movements
        3. variations of players
        4. variation of absolute locations

      Future Work

      • alternative data model
      • simulating set plays
      • validation via video footage





      Clapper, Brian M. and Varoquaux, Gael (2008). „Solve the linear sum assignment problem“. In: url: (visited on Apr. 3, 2021).

      Kelbick, Don (2021). „Paul Westhead’s Loyola Maramount Fast Break and Transition Offense System“. In: url: (visited on Apr. 2, 2021).

      Keogh, Eamonn and Ratanamahatana, Chotirat Ann (2005). „Exact indexing of dynamic time warping“. In: Knowledge and Information Systems 7.3, pp. 358-386.

      Seidl, T., Lames, M., and Wheat, J. (2019). „Radio-based Position Tracking in Sports: Validation, Pattern Recognition and Performance Analysis“. In: Universitätsbibliothek der TU München.

      Sha, Long, Lucey, Patrick, Yue, Yisong, Carr, Peter, Rohlf, Charlie, and Matthews, Iain (2016). „Chalkboarding“. In: IUI’16. Ed. by Jeffrey Nichols, Jalal Mahmud, John O’Donovan, Cristina Conati, and Massimo Zancanaro. New York, New York: The Association for Computing Machinery, pp. 336-47.

      Yanagisawa, Yutaka, Akahani, Jun-ichi, and Satoh, Tetsuji (2003). „Shape-Based Similarity Query for Trajectory of Mobile Objects“. In: Mobile Data Management. Ed. by Ming-Syan Chen, Panos K. Chrysanthis, Morris Sloman, and Arkady Zaslavsky. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 63-77.


      Top 18 ranked plays.


      Some more playbook plays