Industrial Engineering Capstone Team Wins Fourth Place at International Conference for US Open Officiating Project
An undergraduate industrial engineering Capstone team was recognized for their US Open Tennis Championships Electronic Officiating Quality Control project at the Institute of Industrial and Systems Engineers Conference. The team worked with the United States Tennis Association to help automate and standardize the electronic officiating process for the acclaimed tennis event.
An undergraduate Capstone team of four industrial engineering students was recognized at the international Institute of Industrial and Systems Engineers, or IISE, Conference for their U.S. Open Tennis Championships Electronic Officiating Quality Control project. This project was supervised by their Capstone Faculty Advisor, Professor Kris Jaeger-Helton, a professional tennis umpire herself. Across two semesters, Alice Gadon, Emircan Gezer, Jonathan Goldstein, and Anders Narita, all E’24, worked with the United States Tennis Association, or USTA, to help automate and standardize the electronic officiating process at the US Open.
The challenge
(left to right) Goldstein, Jaeger-Helton, Gezer, and Gadon at the 2023 US Open.
A few years ago, the US Open, which is held annually in New York, replaced its line judges with electronic officiating technologies known as Electronic Line Calling Live, or ELC Live. Recently, the US Open made history by scaling up Video Assistant Reviews, or VAR. ELC Live tracks the ball and automates line calling, while VAR allows for footage to be reviewed for other types of rulings during play. The VAR reviews aid decisions on calls related to questions of fact other than line calls, such as double bounces or touches during a point. Although ELC Live removes the need for line judges, it must be monitored and verified by human Review Officials, or ROs, as do the challenged VAR calls. Some ROs, known as VROs, are additionally trained on reviewing the VAR process.
Dr. Jaeger-Helton, also a teaching professor and the Director for the Industrial Engineering undergraduate program at Northeastern, closely advised the Capstone team as they worked on the project and put their deliverables together. She is also the Lead Review Official at the US Open, managing nearly 60 ROs and their work in automated officiating for the Open. She was able to provide valuable insight on the pre-existing processes from her personal work experience.
“While the US Open matches are in progress, I receive reports from Review Officials through a form we’ve created; at the end of their shift, they’ll scan a QR code and submit a report about anything related to electronic officiating that may require attention,” Jaeger-Helton says. “I get reports throughout the day of these incidents and issues and prioritize how to address each situation. Near the end of the day, I need to generate a daily summary and meet with the Referees.”
Due to a reliance on manual adjustments by the Lead RO from RO entries and the recent advent of VAR, these reports can often take more time than is available, especially if there is a high-profile issue or if there were many incidents in a short time span. Jaeger-Helton tasked the Capstone team with helping to streamline and standardize the electronic officiating processes to help manage these issues and the associated records. In addition, further analysis of the data made it possible to reveal patterns across incidents and issues that could be addressed more efficiently by the US Open officiating leadership.
The solution
Gadon, Goldstein, and Gezer at work during the 2023 US Open.
As part of their analysis of the current system, the Capstone team was approved to attend the 2023 US Open in New York to observe the electronic officiating processes, identify metrics, and collect data. While at the tournament, they gathered this information by analyzing ELC Live incidents and VAR scenarios to determine the reporting processes and the various incident outcomes. Data such as where and when incidents occurred, how outward-facing they were, and how they were handled had all been recorded, but the USTA had not previously had this data available in a digestible format.
Once the team identified which aspects of the Lead RO’s system needed to be streamlined, they began compiling a deliverables package that included data forms, statistical analyses, visualization techniques for the embedded data patterns, enhanced reporting on electronic officiating, a map to show how the deliverables interact, and recommendations for preventative and optimization procedures. Attending the US Open gave the team a much clearer vision of their project and how they could accomplish their goals.
“We ended up doing a lot more, and that was facilitated by going to the Open,” Goldstein says. “Our idea of the project expanded into different aspects of RO versus VRO [responsibilities] and making different facilitators along with something that helps them visualize the data.”
The first portion of the deliverables package was the team’s data and incident analysis. The analysis showed that while the total number of ELC Live incidents and number of matches both decreased over time, the number of incidents per match increased. On courts that were monitored by less experienced Technical Provider Operators, the total number of incidents was higher. The team’s VAR data also helped them determine where support could be added and protocols could be altered to reduce the time spent on live video reviews.
Beyond this data analysis, pre-tournament checklists for the ELC Live and VAR processes were designed to minimize potential oversights or errors. These checklists –organized by subsystem components and locations– made it easier to ensure that all communication and electronic officiating equipment was functioning properly before the tournament and at the start of each day of play. The most important deliverable, however, was the Daily ELC Incident Summary Workbook. As the Lead RO, Jaeger-Helton’s responsibility is to find, investigate, address, and then present any incident patterns or significant singularities at the daily officiating meeting. By creating a summary generator powered by VBA automation, the Capstone team was able to computerize key aspects of this process. After some pre-analysis by the Lead RO, the Daily Summary Generator reduced the time the process took from an hour to under 10 minutes. It also provided a standard template with automatically mined information.
The result
Gezer, Gadon, Jaeger-Helton, and Goldstein outside the Arthur Ashe Stadium.
Once these deliverables were completed, the team met with USTA officiating executives to showcase the package and demonstrate the Daily ELC Incident Summary Generator. As a guide for officials who would use the new deliverables, the team created Standard Operating Procedure documentation on the Summary Generator and the new VAR process. The executives approved the package, and the deliverables were officially handed over to US Open Chief Umpire Andrew Walker in April, who was also the sponsor for this initiative.
For their hard work and dedication, the Capstone team earned the Most Deliverable Award at the Industrial Engineering Capstone Awards. They also shared the achievement of creating the first-ever externally initiated data and process improvement officiating project across the USTA. By contributing to modern VAR technology and its processes, the team broke new ground in the world of professional tennis.
“VAR is a technology that was being used in other sports, but that was the first time [it was scaled up] in tennis,” Gezer says. “We were able to create supportive facilitators and checklists for that, so [we learned that] that we made history.”
In May, IE senior Alice Gadon headed to the IISE Conference in Montréal to represent the Capstone team and present their project. The selection process to attend was rigorous, and then the team made it to the finals and officially placed fourth in the Capstone Senior Design Project Service Systems category. They also received a certificate which recognized their “outstanding application of ISE knowledge and skills with a significant impact on an organization.” Gadon says she was proud to present the project because of how it reflects the accomplishments of industrial engineers.
“Ours was showcasing a real-world application of industrial engineering from a consulting point of view, so a bit more of the tangible stuff that we learned in class,” Gadon says. “It was a really awesome experience.”
The team is excited to see their accomplishments in use at future US Opens, including the upcoming 2024 Open in late August. For them, the most fulfilling aspect was making a long-lasting impact with their work by solving problems outside the classroom.
“I think we were able to create something really unique that can be used in the future,” Narita says. “That’s really special, and it’s a big part of what made this project worth it.”