VIRTUAL UMPIRE SYSTEM

Description

RELATED APPLICATIONS AND CLAIM OF PRIORITY

This patent document claims priority to U.S. Patent Application No. 63/701,902, filed Oct. 1, 2024. The disclosure of the priority application is fully incorporated into this document by reference.

BACKGROUND

The number of sporting events in public or community-owned fields and parks continues to grow. The demand for courts, fields, and other playing surfaces for individual and team practices, scheduled matches, and impromptu recreational games is outpacing the resources available to support such activities. One such resource is umpiring. Many sporting activities, including baseball, softball, cricket, soccer, football, lacrosse, tennis, pickleball, and others require impartial judges to determine whether or not the location of a ball is inside a defined area in the field of play such as a strike zone, foul line, or court boundary. However, the availability of impartial umpires to officiate such events cannot fully meet the needs of these activities. And when a volunteer or inexperienced umpire watches an event, the umpire's calls can be prone to errors, biases, and limitations on visibility.

Virtual umpire systems such as the Automated Ball-Strike (ABS) system used by Major League Baseball (MLB) are available. This system, known as the Hawk-Eye system, uses a group of cameras to track a pitched ball and compare the location of the ball to the batter's strike zone. The MLB previously used a radar-based version of the system. According to various news reports, the MLB, but switched to the camera-based system to seek improved accuracy. In addition, in both radar-based and camera-based systems, the previous systems require expensive, specialized equipment to be installed at particular locations of a baseball stadium. Such installations are not technically possible and/or financially feasible for community field owners, operators, and users.

Accordingly, virtual umpire systems that are accurate, cost-effective, and installable in outdoor fields and parks that lack the infrastructure of a professional sports team's stadium are needed. This document describes methods and systems that are directed to addressing at least some of these issues.

SUMMARY

In some embodiments, this document describes a virtual umpire system includes multiple fixtures positioned proximate to a playing surface. Each fixture includes: (a) a support; (b) a radar system connected to and supported by the support and positioned to capture radar data for an area that includes the playing surface; and (c) a local controller stack. The stack includes control circuitry for controlling the radar system and a instructions that will, upon execution, cause a processor to, in response to receiving a command to provide a virtual umpire service: (a) cause the radar system to capture the radar data for the area during a time period; (b) process the radar data to identify, in the radar data, a specified object; (c) determine whether a position of the specified object is within or outside of a boundary of the playing surface; and (d) send a result of the determining to another device of the system.

In some embodiments, a method of providing a virtual umpire service includes actions taken a processor of a local controller stack that is connected to a fixture that includes a radar system that is positioned to capture radar data for an area that includes a playing surface. The processor will, in response to receiving a command to provide the virtual umpire service: (a) cause the radar system to capture the radar data for the area during a time period; (b) process the radar data to identify, in the radar data, a specified object; (c) determine whether a position of the specified object is within or outside of a boundary of the playing surface; and (d) send a result of the determining to another device of the system.

Some embodiments are directed to a computer program product that will, upon execution, cause a processor of a fixture that includes a radar system, in response to receiving a command to provide a virtual umpire service to perform any of the actions that such a processor may perform in any of the variations of the embodiment listed in the previous paragraph.

In some embodiments, an umpire-as-a-service system includes a radar system and a controller operably connected to the radar system. The controller includes control circuitry for controlling the radar system, along with a memory containing local programming instructions that will, upon execution when the radar system is positioned to capture radar data for a playing surface, cause a processor of the system to, in response to receiving a command to provide a virtual umpire service: (a) cause the radar system to capture the radar data for the area during a time period; (b) process the radar data to identify, in the radar data, a specified object; (c) determine whether a position of the specified object is within or outside of a boundary of the playing surface; and (d) send a result of the determining to another device of the system.

In some embodiments, an umpire-as-a-service system may include a computer program product with programming instructions that will, upon execution, cause a processor to, in response to receiving a service request from a mobile electronic device: (a) identify a radar system that corresponds to the service request, wherein the radar system is positioned to capture radar data for an area that includes a playing surface to which the mobile electronic device is proximate; and (b) cause the mobile electronic device to output a user interface that includes a user-selectable prompt to trigger operation of a virtual umpire service. In response to receiving both a user selection of the prompt and payment information for the virtual umpire service via the user interface: the processor may generate a command to operate the virtual umpire service by causing the radar system to capture radar data for the area that includes the playing surface for the area, and the processor may send the command to a controller that is communicatively connected to the radar system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates example elements of a combined outdoor lighting and virtual umpiring system.

FIG. 2 illustrates actions that may be implemented in a method of controlling activation of outdoor lighting and virtual umpiring services.

FIG. 3 illustrates an example client electronic device user interface for controlling operation of outdoor lighting equipment and virtual umpiring services.

FIG. 4 illustrates example elements of an administrator user interface.

FIG. 5 illustrates hardware components that a local controller stack of a fixture may include.

FIG. 6 illustrates example components of an electronic device that may be used in various elements of the system.

FIG. 7 is a flow diagram illustrating various elements of operating a virtual umpire service.

FIG. 8 illustrates an example system topology for a virtual umpire service.

DETAILED DESCRIPTION

In this document, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. Unless defined otherwise, all technical and scientific terms used in this document have the same meanings as commonly understood by one of ordinary skill in the art. As used in this document, the term “comprising” (or “comprises”) means “including (or includes), but not limited to. ”

Additional terms that are relevant to this disclosure will be defined at the end of this Detailed Description section.

This disclosure relates to methods and systems that enable any user of a playing surface to access a virtual umpiring service while the user is practicing or playing a sport on the field. As used in this document, the term playing service refers to a sporting field, court, pitch, or other area on which a sports game or practice is performed. Playing services may thus include baseball, softball, and cricket fields; basketball, tennis, pickleball, and paddleball courts; football, lacrosse, rugby, and soccer fields or pitches; and other locations on which a sport is played or practice.

Various embodiments include a virtual umpire system integrated into a modular sports lighting system that allows any person having an internet-connected mobile device who visits a playing surface to access a virtual umpire service at the field for a specified duration. Additionally, in some embodiments, the system may enable users to access additional functions such as lighting services and/or camera footage of the field at which the system is installed.

FIG. 1 illustrates example elements of an example combined virtual umpire and outdoor lighting system. The system includes one or more fixtures 110, each of which comprises a support 112 such as a pole, tower, stand, yoke mount, or other structure that holds, supports, and positions above ground one or more luminaires 114 and radar systems 118.

Each luminaire 114 includes one or more groups of light emitting diode (LED) or other illumination-producing modules. The luminaires 114 may be arranged to extend laterally from the support 112 as shown in FIG. 1, or they may be arranged in a different configuration.

To provide a virtual umpire service, a radar system 118 that includes an antenna, transmitter, receiver, and associated signal generation and processing circuitry will be attached to the fixture 110 and positioned to capture information about the playing surface 120. The radar system 118 may be attached to the support 112, any of the luminaires 114, and/or the local controller stack 116. Multiple radar systems 118 will be installed on various two or more fixtures 110 positioned at various locations near the playing surface to capture radar data for any location of the playing surface from multiple data capture points and angles.

A local controller stack 116 is attached to the support 112 and includes one or more weather-resistant enclosures, each of which contains a power supply, control circuitry for controlling the one or more luminaires of the fixture, and memory. Each local controller stack 116 transforms local electricity from alternating current (AC) to direct current (DC) power and includes LED drivers for operating the luminaires 114 and radar systems. The controller stack 116 may receive AC power from an external power source such as a power cable. The power supply may may transform the AC to DC for delivering DC to the luminaires 114, the radar system 118, and/or a rechargeable battery. Optionally, the power supply may include a solar panel that transforms solar energy into DC for operating the luminaires 114, operating the radar system 118, and/or charging the battery. Additional elements of an example local controller stack 116 are disclosed in FIGS. 15-19 and the corresponding description of U.S. Pat. No. 11,371,690, the disclosure of which is fully incorporated into this document by reference.

Each fixture 110 may, when installed at a facility that includes a playing surface 120 of an outdoor facility, illuminate the playing surface when its luminaires 114 are activated. Additional features about example fixtures 110 will be described below and are also described in U.S. Pat. No. 11,371,690 and U.S. Pat. No. 12,104,777, the disclosures of which are fully incorporated into this document by reference.

In various embodiments, the system may include a gateway controller 131 that is positioned at the facility that includes the playing surface 120. The gateway controller 131 is an electronic device that is communicatively connected to each of the local controller stacks 116 of the fixtures 110 and radar systems 118 at the playing surface via a local area network, a wired or local wireless connection, or other communication system. The gateway controller 131 will include a processor and a memory, with programming that enables the gateway controller 131 to send commands to control, and receive information from the local controller stacks 116 of the fixtures 110, the radar systems 118, and other electronic devices of the system.

The system also may include a remote server 141 that is positioned either at the facility that includes the playing surface, or away from the facility at another location in a cloud-based arrangement. The remote server 141 may communicate directly with the local controller stacks 116 and radar systems 118, or it may indirectly communicate with the local controller stacks 116 and radar systems 118 by sending commands to and receiving information from the local controller stacks 116 and radar systems 118 via the gateway controller 131, which will relay communications to and from the local controller stacks 116 and radar systems 118.

A client mobile electronic device 133 will enable a user to control operation of the luminaires 114 and radar systems 118 at the playing surface 120 using methods described in this document. The client mobile electronic device 133 will include: (a) a display; (b) a camera that can capture an image of a quick response (QR) code or other barcode 121 that is displayed by a sign, label, or display that is on or near the fixture 110; (c) a browser or other software application that can access a site or portal hosted by the server 141 at an address contained in the barcode 121; and (d) communication hardware for communicating with the server 141 and/or the gateway controller 131.

An administrator electronic device 135 may be included and communicatively connected with the server 141 and/or the gateway controller 131. The administrator electronic device 135 will allow an authorized administrator such as an operator of the facility that includes the playing surface 120 to manage operation of the luminaires 114 and radar systems 118 at the facility. While the administrator 135 must provide credentials indicating that the administrator is authorized to manage operation of the facility, a client mobile electronic device 133 will not necessarily need to provide credentials or other authorization in order to control one or more of the fixtures 114.

FIG. 2 illustrates methods by which the system will allow a user to control operation of one or more fixtures at a facility. At 201 a client electronic device will access a server at which the virtual umpire service, and optionally other services such as a lighting activation service, is administered. The client electronic device may do this by using a dedicated software application on the client device, or by accessing a website of the service by entering a web address or capturing an image of a barcode that is displayed on a sign, label, or display at a facility and that includes an address of a server for the service, optionally along with an identification code for the fixture or modules at the facility to which the barcode is attached or to which the barcode is most proximate.

At 202 the client electronic device will request the service by pointing the browser or other application to the address of the server to request the virtual umpire service and/or the lighting service. This will cause the server to return (at 203) , and the client electronic device to output, a user interface for the service. An example user interface 301 will be described in more detail below in the discussion of FIG. 3. If the barcode includes an identification code for a particular playing surface, fixture, or group of fixtures (such as all lights and/or radar systems that are directed to a particular playing surface), the server may select and return information for that fixture or group. Alternatively, the server may cause the user interface to output a set of candidate fixtures, and a user of the client electronic device may select any of the candidate fixtures for operation.

At 203, in response to the service request, the server will transmit user interface information, such as a web page or data that an application installed on the client device that includes candidate operational parameters that a user of the client device may select for controlling operation of the lighting and/or virtual umpire services for the devices that are associated with the barcode. The candidate operational parameters may include, for example: (a) time of operation such as a start time, stop time, and/or duration; and/or (b) operational settings for the luminaires (such as brightness or color temperature). Examples of these are illustrated in FIG. 3, in which the candidate parameters include light level 302 and duration of the lighting and/or virtual umpire service 303. Optionally, the operational parameters may be associated with one or more scenes, each of which is a group of operational parameters that a luminaire or set of luminaires may use to emit light over a period of time. A scene may remain static over the period of time, or it may change such as by causing the LEDs to flash, increase or reduce intensity, change colors, or otherwise change one or more characteristics of light output by the luminaire over time.

The user may use the client electronic device to select operational parameters from the candidate parameters, and at 204 the server will receive the selected operational parameters from the client device.

At 205 the user interface may will offer a user the ability to activate the virtual umpire service, as well as other services such as a lighting service and/or a camera footage capture service. For example, the user interface of FIG. 3 also indicates that the user interface may offer the user the ability to turn on the virtual umpire service 307 for the playing surface during the period of time that has been selected using the operational parameters.

If the fixture 110 also includes an attached camera, the user interface may offer the user an ability to turn on a video capture service 306 that, when activated, will cause a camera that is attached to the fixture to capture digital video of the playing surface while the virtual umpire service is on, and/or when implementing a lighting service request. The local controller stack of the fixture will receive the digital video and send it to the server and/or the local gateway, which will store the digital video for retrieval and download by the user. The system may send the user a link to the storage location, or otherwise provide the user with access to the video, and the system may delete the video after the user retrieves it or after a threshold period of time has elapsed. The inclusion of video footage captured by a camera can increase users' interest in operating the system by allowing for the recording of activities that happen at the field during the purchased lighting session.

At 206, the server may then wait for the client device to provide payment information (304 in FIG. 3) such as credit or debit card information, digital wallet information, payment application information, or other account information via which the user will pay the operator for activating the virtual umpire service and/or other services. In response to receiving the operational parameters and payment information, at 210 the server will generate a command with instructions to automatically activate the virtual umpire service for the selected fixture(s) according to the selected operational parameters.

Optionally, one or more remote, third-party services (140 in FIG. 1) may operate to send data to the local controller stack that speakers, microphones, and/or other hardware of the local controller stack may be available at the facility at which a virtual umpire service is activated. Example remote, third-party services 140 include a severe weather alert service, a voice-triggered emergency service that will contact law enforcement or medical services in response to hearing a particular command phrase, and/or a streaming music service that may play audio via a speaker that is attached to the fixture. Some such services (such as a streaming music service) may be available to the user for an additional fee. Other such services (such as severe weather and/or emergency services) may automatically be provided when the user purchases a virtual umpire service, or a minimum level of service such as that for at least a threshold period of time. In embodiments that receive severe weather alerts and/or other emergency alerts from the third-party service 140, the additional services may be operated at 211 by outputting an audio prompt of the alert, and/or causing the fixtures to implement a scene that is associated with the alert, such as one that causes the lights of the luminaires to flash, change color, and/or change intensity.

As noted above, in some embodiments the operational parameters that are available for a user to select may be associated with one or more scenes for operating one or more luminaires over a period of time. In various embodiments, scenes that a particular fixture can implement may be stored on the local controller stack of the fixture. If so then when the server receives a user interface request for a fixture, the server may: (a) access a data set that associates available scenes with fixture; (b) retrieve, from the data set, the available scenes for the fixture; and (c) present the user with operational parameters for those scenes. Then, when generating the command for the luminaires to operate, the server may generate instructions for the fixture to trigger one of the scenes that corresponds to the user-selected operational parameters. By storing the scenes at the local controller stack, this can reduce latency in communication since it only requires a limited initial command to activate the scene, and it does not require the fixture to maintain any communication with the server during operation.

Notably, in some embodiments the system does not necessarily require the user to set up an account, download a dedicated application, or otherwise be authorized to operate the fixtures. Instead, the user simply needs to be present at the facility with a mobile electronic device that can capture the barcode, select time or other parameters for operation, and provide payment. This provides advantages over prior systems that either restrict usage to authorized users, or which require specialized hardware such as lighting activation kiosks at the facility.

However, even in embodiments that do not require user preauthorization, the system still provides security and management capabilities by allowing an administrator to limit or override actions that any of the fixtures may perform at given periods of time. The administrator electronic device 135 will output an administrator user interface by which the administrator may exercise control over lighting and virtual umpire systems and limit individual user activations. FIG. 4 illustrates example elements of one screen of an administrator user interface 400. The system may only allow the administrator user interface to operate on the administrator electronic device after the system receives an administrator credential such as username and password, biometric identifier, or other required authorization token. The user interface may include user-selectable fields by which administrators may define and schedule events 401 to take place at a facility. An event will include an identifier for a facility and/or gateway controller of a facility 402, one or more fixtures 403 at the facility, and operational parameters 405 for the event such as what condition will trigger the event, whether the event is temporary or recurring, and/or whether the event will be scheduled at a particular time.

Returning to FIG. 2, if an administrator has scheduled an event at the facility, then before causing the system to implement the user's request for service at 210, at 207 the system will first determine whether an administrator is exercising control over the fixtures at the facility, such that the user's request conflicts with the administrator's scheduled event. If there is a conflict, such as if an event is scheduled at the present time or during the user's requested duration of operation (207: YES), then at 208 the system will limit the service that the user can receive. For example, the system may prevent any change in operation of the system, or any fixtures that are part of the event, by not sending operational commands to implement the user's request until the event is complete.

In addition, during a time when the system is causing radar systems and/or luminaires of fixtures to operate according to user requests, if the system receives an override request (215: YES) from an administrator computing device, it may limit the service 208 as shown above until an event associated with the override request is complete. To do this, the system may generate and send a command to the fixtures to operate according to the administrator request instead of the user-selected operational parameters.

In various embodiments, as noted above each fixture includes comprises a local controller stack. FIG. 5 illustrates example elements of a local controller stack. The elements shown in FIG. 5 may be contained in a single enclosure 501 or distributed among multiple enclosures. The local controller stack's processor 502 will use programming instructions stored in memory a 507, sensor data received from one or more external sensors 505 such as a camera or radar system via a sensor input 504, and commands and parameters received from external devices (such as a server or gateway controller) via a communication circuit 508 such as a communications port or receiver to generate signals that will cause light-emitting modules, such as LED modules 535, to operate. Each LED module may have an associated LED driver 531a . . . 531n that will receive the signals, along with power from the local controller stack's power circuit, to operate the LED modules.

The power circuit may include a power conditioning and/or surge protection circuit 511 with components such as an electromagnetic interference (EMI) filter. An AC/DC converter 513 will convert AC received from an external power source 515 to DC. In addition or alternatively, a DC/DC converter may convert DC received from a battery or other power source to a level that will operate the LED drivers. The power circuit will deliver electricity to each LED driver 531a . . . 531n for powering the LED modules 535.

Each conductive path leading from the power circuit to an LED driver may include an energy usage sensor 532a . . . 532n that serves as a power monitor to measure the energy used by each LED module. Example power monitors include current sensors, current transformer (CT) sensors, voltage transformers, and watt meters. Optionally, the power circuit also may include a master energy sensor that serves as an overall input power monitor 512 to measure energy used by all LED modules that the local controller stack operates. When the processor receives, from any of the power monitors, a measurement of energy used by the one or more luminaires of the fixture when operating according to the selected one or more operational parameters, the processor 502 will send the measurement of energy to the server via the communication circuit 508.

Optionally, the administrator electronic device and/or server may use the energy measurements to control various aspects of operation of the system. For example, returning to FIG. 2, when the server receives an umpire service request and/or lighting service request, the server may access real-time energy price information, and the server may calculate a cost for the requested service that is a function of the real-time energy price. For example, if the time of the service request is associated with a time when the real-time energy price is higher or lower than a baseline energy price, the server may modify the cost by multiplying it by an adjustment factor that is a function of the difference between the real-time energy price and the baseline energy price. In addition, if time of the service request coincides with a time at which an energy usage restriction is in place, the system may apply administrator control at 207 to reduce energy used by the LED modules if the measured energy is higher than a threshold amount, such as within a minimum distance from a peak energy level. Options for reducing energy may be to limit the requested service at 208 by reducing brightness of the LEDs, permitting only a subset of the LEDs and/or radar systems to operate, otherwise limiting use of the radar system, or restricting access to additional functions such as cameras or loudspeakers that are attached to the fixtures.

FIG. 6 depicts an example of internal hardware that may be included in any of the electronic devices of the system, such as the client mobile electronic device, administrator electronic device, server, and/or local controller stack. A conductive path such as that of one or more circuit boards 601 serves as a communication path via which messages, instructions, data, or other information may be shared among the other illustrated components of the hardware. Processor 605 is a central processing device of the system, configured to perform calculations and logic operations required to execute programming instructions. A memory 610 is a non-transitory device or collection of devices across which data and/or instructions are stored.

The electronic device may display information processed by the processor 605 on a video output device 620 (i.e., a display device) in visual, graphic, and/or alphanumeric format. Example video output devices include display screens, heads-up displays such as those used in virtual reality and augmented reality devices, projection devices, and video output ports such as HDMI ports that transfer information to an external display device. An audio output device 615 such as a speaker, video output port, or wireless transceiver such as those that operate via a Bluetooth® or other wireless communication protocol. Communication with external displays, audio outputs, and electronic device also may occur using various communication circuit devices 630 such as a wireless antenna, a radio frequency identification (RFID) tag and/or short-range or near-field communication transceiver, each of which may optionally communicatively connect with other components of the electronic device. The communication circuit devices 630 may be configured to be communicatively connected to a communications network, such as the Internet, a local area network or a cellular telephone data network.

The electronic device also may also include a user interface device 635 that includes one or more input devices that can receive data and/or commands from a user. Example user interface devices 635 include a keyboard, a mouse, touchscreen that is part of a display device of the video output 620, a touch pad, a remote control, a pointing device, and/or a microphone.

A camera 640 will include image sensors and other hardware that can capture video and/or still images.

The electronic device also may include or be electronically connected with a radar system 650 that includes an antenna 651, transmitter 652, receiver 653 and associated signal generation and processing circuitry.

The electronic device also will include a power module 660 that, in operation, will provide power to the other components of the electronic device. The power module 660 may include an internal battery, a power port that will receive power when electrically connected to an external power source, a power management integrated circuit (PMIC), a DC-to-DC converter or AC-to-DC converter, a receiving coil for inductive charging, other power circuit components, and/or a combination of any of these.

Based on the teachings contained in this disclosure, it will be apparent to persons skilled in the relevant art(s) how to make and use embodiments of this disclosure using data processing devices, computer systems and/or computer architectures other than that shown in FIG. 6.

FIG. 7 illustrates additional example elements of methods of operating the virtual umpire system. At 701 the system will define one or more boundaries associated with the playing surface. Example boundaries may include that of a strike zone positioned over home plate of a baseball or softball field, foul lines of a baseball or softball field, The system may retrieve this data from a database of boundary coordinates on the playing surface, by accessing previous records of video data of the playing surface, by receiving user definitions of the boundaries, or by other methods.

At 702 the system will capture radar data captured by radar systems positioned at various locations at or near the playing surface, and at 703 the system will initially process the radar data to identify a boundary 713 of a target area and a target object 714 in the radar data.

The system may identify the boundary at 713 using any suitable data processing technique. For example, if the boundary is a foul line or a court out of bounds line, the system may process the radar data and use known reference coordinates to identify line segments that correspond to boundaries in the radar data. In other embodiments, the system may look for a radar reflective marker that is positioned on part of the boundary, and the system may place the boundary at a location of the marker and/or use the marker as a reference point and establish the boundary as a line extending a predetermined distance from the boundary. Example markers may include, for example, a foul line pole of a baseball field on which a corner reflector, a spherical retroreflector, or a phase-conjugate mirror is installed.

To determine a boundary that may vary at different points in time, the system may look for reference points in the radar data and define the boundary with respect to the reference points. For example, to define a strike zone of a batter in a baseball or softball game, the system may identify the batter and the field's home plate in the radar data, it may calculate the strike zone as a rectangular area over the home plate that extends between two specified locations on the batter's body (such as from a midpoint between the batter's shoulders and belt line to the bottom of the batter's knees), with a width corresponding to that of the home plate. Optionally, if the fixture also includes a camera, the system may capture video of the playing surface, use edge detection or another image processing technique to identify the boundary, and apply an image layer with the boundary over the radar data layer to map the boundary location in the image data to the radar data.

To identify a location of target object 714 such as a ball or other moving sports object in the radar data, the system may use any suitable radar data processing algorithm, such as by identifying an object track in the data for the target object as it moves. Optionally, the object may include an integrated radar reflective marker, and if so the radar system can use the marker to help identify the object.

Either or both of steps 713 and 714 may use any now or hereafter known radar data processing algorithms. In addition or alternatively, these steps may process the radar data using a classifier (a machine learning model) that has been trained to recognize boundaries and/or track objects in radar data.

Notably, the initial processing of step 703 (including substeps 713 and 714) is performed at the local controller stack of the fixture to which the radar system is attached, thus providing an edge computing arrangement in which a significant amount of the process is at the location where the radar data is captured, to provide a localized measurement of the object's trajectory and/or position with respect to the boundary.

As and after the initial processing is performed, at 704 one or more of the local controller stacks, or the gateway controller, or a remote server will use triangulation, time synchronization, and/or other methods such as those described below to process collective data for all of the radar systems that are collecting data of the playing surface, including the results of the individual radar systems'processing (at 703) to refine the boundary identification 723 and the target object identification 724. The data processing will help the system provide a more accurate measurement of whether the object is within or outside of the boundary at any given point in time. Example refining steps may include comparing results from all radar systems and discarding outliers or inconclusive results, generating averages of the results from one or more individual radar systems, and/or triangulation the data (i.e., by taking the results received at two or more of the individual radar systems locations and measuring the angle and/or difference between the data points captured in the two systems).

At 708 the system will output a result of the determination to another electronic device, such as the gateway controller, the user electronic device, or other devices that are in communication with the system. The result may be, for example, an indicator of whether a pitch is a ball (outside of the strike zone) or strike (in the strike zone), whether a hit ball is inside or outside of a foul line, and an indication of whether a football or soccer player's foot is inside our outside the boundaries of a field or pitch.

At 709 the system may use the result to activate other communicatively connected devices to perform ne or more functions. For example, the system may cause luminaires at the playing surface to flash, change color, change brightness, or otherwise change characteristics of light that the luminaires are emitting if a pitch is a strike (or a ball), if a ball moves beyond an out of bounds line or foul line, or if player steps out of bounds. As another example, the system may send a message to scoreboard control system that will cause scoreboards at the facility to update information displayed on the scoreboard based on the determination at 707, such as by updating a ball count or strike count for a current batter.

FIG. 8 illustrates an example system topology for a playing surface in which a gateway controller 800 and various fixtures 801 . . . 806 are positioned at various locations over or near the playing surface. Each fixture includes a radar sensor, a local controller stack to provide edge computing capabilities, and optionally one or more luminaires that will illuminate the playing surface when activated. The local controllers of fixtures 801 . . . 806 may communicate with each other and/or the gateway controller 800 using any communication protocol, such as that of a wireless mesh network. A mesh network can allow node-to-node data sharing, reduced redundancy, and enhanced distributed decision-making, thus enhancing accuracy of the line of sight from one or more of the radar systems to the target object is obstructed.

For the gateway controller or server to process the collective radar data at 704, the device may integrate time synchronization, triangulation, and error correction into its distributed edge-mesh architecture. Unlike conventional single-point or camera-based solutions, the system described in this leverages N or N+1 radar sensors mounted on fixtures such as new or existing light poles, each equipped with edge computing in the form of the local controller stack. The nodes perform localized detection and pre-processing, while the gateway computing device distributes timing, aggregates results, and interfaces with other systems at the facility that includes the playing surface. Collectively, these N nodes work together as a cooperative network, ensuring redundancy, scalability, and resilience for every officiating decision across baseball and softball fields of varying configurations. These nodes work together as a cooperative network, ensuring redundancy and resilience for every officiating decision.

The local controller stacks and gateway controller may provide any or all of the following functions:

• • Edge Estimates: Each radar system produces a localized measurement of the ball's or other target object's trajectory. By allowing every fixture's radar system to contribute, the system achieves full-field coverage and removes blind spots, ensuring that no single view dictates the outcome.
  • Time Synchronization (PTP+GNSS PPS): Precise synchronization ensures that measurements across all poles are aligned within microseconds. The gateway decive may distribute Precision Time Protocol (PTP) as the primary sync, while GNSS Pulse Per Second (PPS) provides a fallback. This dual system is unique for the sustem's ability to remain reliable even if the network is disrupted.
  • Time Align Block: This block corrects for minor timing offsets, aligning detections on the same clock domain. Accurate time alignment is essential so triangulation can be calculated from simultaneous data, yielding real-time calls with confidence.
  • Triangulation: Using multilateration across the data captured by each of the radar sensors uses distances or timing offsets from multiple sensors then fuses the synchronized data to compute a precise 3D target object trajectory. With N≥3 nodes, the system can solve for the target object's exact 3D location even if some of the nodes have errors in their retuned data and/or results, by using redundant distance measurements. This helps to improve accuracy in strike zone determinations, foul line judgments, player-base interactions, and other event-specific analyses of the captured radar data.
  • Error Correction: To withstand noisy environments, the system may integrate filtering and outlier rejection in the radar data. This can further enhance accuracy, so decisions can be trusted by players, coaches, and fans alike.
  • 3D Pose Output: The output of the system may be a fused 3D position and velocity vector. This data may then be output not a user and/or modules of other facility hardware (such as a scoreboard) and may be shared with pother systems at the facility.

By combining distributed edge computing, dual-mode timing synchronization, and consensus-based triangulation, the system can deliver a high level of accuracy and resilience not possible with prior art systems.

In addition to the patents incorporated by reference above, this patent document also incorporates the disclosures of the following patents by reference in their entirety: U.S. Pat. Nos. 11,644,193, 11,209,153; and 11,284,492.

The following paragraphs provide additional information about various terms used in this document:

In this document, when terms such as “first” and “second” are used to modify a noun, such use is simply intended to distinguish one item from another and is not intended to require a sequential order unless specifically stated.

The term “approximately” when used in connection with a numeric value, is intended to include values that are close to, but not exactly, the number. For example, in some embodiments, the term “approximately” may include values that are within +/−10 percent (or, in some embodiments, +/−5 percent, +/−3 precent, or +/1 percent) of the value. By way of example, the phrase “approximately 100%” will include values from 90% to 100%, and the phrase “approximately 90 degrees” will include values from 81 degrees to 99 degrees.

When used in this document, terms such as “top” and “bottom,” “upper” and “lower”, or “front” and “rear,” are not intended to have absolute orientations but are instead intended to describe relative positions of various components with respect to each other. For example, a first component may be an “upper” component and a second component may be a “lower” component when a device of which the components are a part is oriented in a first direction. The relative orientations of the components may be reversed, or the components may be on the same plane, if the orientation of the structure that contains the components is changed. The claims are intended to include all orientations of a device containing such components.

The term “substantially,” when used in connection with a value, is intended to mean approximately, within a threshold tolerance that is a percentage corresponding to any of the percentages described in the previous paragraph. For example, items described as “substantially the same,” “substantially equal,” or “substantially planar,” may be exactly the same, equal, or planar, or may be the same, equal, or planar within acceptable variations that may occur, for example, due to manufacturing processes and/or tolerances.

In this document, the term “connected”, when referring to two physical structures, means that the two physical structures touch each other. Devices that are connected may be secured to each other, or they may simply touch each other and not be secured.

In this document, the term “electrically connected”, when referring to two electrical components, means that a conductive path exists between the two components. The path may be a direct path, or an indirect path through one or more intermediary components.

An “electronic device” or a “computing device” refers to a device or system that includes a processor and memory. Each device may have its own processor and/or memory, or the processor and/or memory may be shared with other devices as in a virtual machine or container arrangement. The memory will contain or receive programming instructions that, when executed by the processor, cause the electronic device to perform one or more operations according to the programming instructions. Examples of electronic devices include personal computers, servers, mainframes, virtual machines, containers, gaming systems, televisions, digital home assistants and mobile electronic devices such as smartphones, fitness tracking devices, wearable virtual or augmented reality devices, Internet-connected wearables such as smart watches and smart eyewear, personal digital assistants, cameras, tablet computers, laptop computers, media players and the like. Electronic devices also may include components of vehicles such as dashboard entertainment and navigation systems, as well as on-board vehicle diagnostic and operation systems. In a client-server arrangement, the client device and the server are electronic devices, in which the server contains instructions and/or data that the client device accesses via one or more communications links in one or more communications networks. In a virtual machine arrangement, a server may be an electronic device, and each virtual machine or container also may be considered an electronic device. In the discussion above, a client device, server device, virtual machine or container may be referred to simply as a “device” for brevity. Additional elements that may be included in electronic devices are discussed above in the context of FIG. 6.

The terms “processor,” “processing device,” and “controller” refer to electronic device hardware that is configured to execute programming instructions. These terms may refer to a single processing device or any number of processing devices in a set of processors that collectively perform a set of operations. Thus, unless the context specifically states that a single processor or controller is required or that multiple processors or controllers are required, the terms “processor” and “controller” include both the singular and plural embodiments. Example processing devices include, a central processing unit (CPU), a graphics processing unit (GPU), a remote server, or a combination of these.

The terms “memory,” “memory device,” “computer-readable medium” and “data store” each refer to a non-transitory device on which computer-readable data, programming instructions or both are stored. Read only memory (ROM), random access memory (RAM), flash memory, hard drives and other devices capable of storing electronic data constitute examples of memory devices. A “computer program product” is a combination of a memory device and the programming instructions stored in it. Unless the context specifically states that a single device is required or that multiple devices are required, the terms defined in this paragraph include both the singular and plural embodiments, as well as portions of such devices such as memory sectors.

A “server” is a computing device that includes one or more processors and one or more memory devices.

In this document, the terms “communication link,” “communication path,” and “communication circuit” mean a wired or wireless path or paths and associated hardware via which a first device sends communication signals to and/or receives communication signals from one or more other devices. Devices are “communicatively connected” if the devices are able to send and/or receive data via a communication link. “Electronic communication” refers to the transmission of data via one or more signals between two or more electronic devices, whether through a wired or wireless network, and whether directly or indirectly via one or more intermediary devices. The network may include or is configured to include any now or hereafter known communication networks such as, without limitation, a BLUETOOTH® communication network, a Z-Wave® communication network, a wireless fidelity (Wi-Fi) communication network, a ZigBee communication network, a HomePlug communication network, a Power-line Communication (PLC) communication network, a message queue telemetry transport (MQTT) communication network, a MTConnect communication network, a cellular network a constrained application protocol (CoAP) communication network, a representative state transfer application protocol interface (REST API) communication network, an extensible messaging and presence protocol (XMPP) communication network, a cellular communications network, any similar communication networks, or any combination thereof for sending and receiving data. As such, a network may be configured to implement wireless or wired communication through cellular networks, WiFi, BlueTooth, Zigbee, RFID, BlueTooth low energy, NFC, IEEE 802.11, IEEE 802.15, IEEE 802.16, Z-Wave, Home Plug, global system for mobile (GSM), general packet radio service (GPRS), enhanced data rates for GSM evolution (EDGE), code division multiple access (CDMA), universal mobile telecommunications system (UMTS), long-term evolution (LTE), LTE-advanced (LTE-A), MQTT, MTConnect, CoAP, REST API, XMPP, or another suitable wired and/or wireless communication method.

In this document, the term “camera”, when used in connection with a camera of a mobile electronic device generally refers to a hardware sensor that can acquire digital images. A camera may capture still and/or video images, and optionally may be used for other imagery-related applications. For example, a camera can be held by a user such as a DSLR (digital single lens reflex) camera, cell phone camera, or video camera. The camera may be part of an image capturing system that includes other hardware components.

The term “barcode”refers to a pattern or symbol that contains encoded data. Barcodes may include one-dimensional barcodes, two-dimensional barcodes (such as matrix codes, Quick Response (QR) codes, Aztec codes and the like), or three-dimensional barcodes.

The term “classifier” means an artificial intelligence model that has been trained to assign a label or category to one or more data points in a set of data that is entered into the model. A classifier includes an algorithm that is trained via an automated process such as machine learning. A classifier typically starts with a set of labeled or unlabeled training data and applies one or more algorithms to detect one or more features and/or patterns within data that correspond to various labels or classes. The algorithms may include, without limitation, those as simple as decision trees, as complex as Naïve Bayes classification, and/or intermediate algorithms such as k-nearest neighbor. Classifiers may include artificial neural networks (ANNs), support vector machine classifiers, and/or any of a host of different types of classifiers. Once trained, the classifier may then classify new data points using the knowledge base that it learned during training. The process of training a classifier can evolve over time, as classifiers may be periodically trained on updated data, and they may learn from being provided information about data that they may have mis-classified. A classifier will be implemented by a processor executing programming instructions, and it may operate on large data sets such as image data, radar system data, and/or other data.

The features and functions described above, as well as alternatives, may be combined into many other different systems or applications. Various alternatives, modifications, variations or improvements may be made by those skilled in the art, each of which is also intended to be encompassed by the disclosed embodiments.

As described above, this document discloses system, method, and computer program product embodiments. The system embodiments include a local computing device, which may have access to one or more remote computing devices. In some embodiments, one or more of the remote computing devices also may be part of the system. The computer program embodiments include programming instructions, stored in a memory device, that are configured to cause a processor to perform the methods described in this document.

The following paragraphs further summarize various embodiments of the methods and systems disclosed in this document.

In a first embodiment, this document describes a virtual umpire system that includes multiple fixtures positioned proximate to a playing surface. Each fixture includes: (a) a support; (b) a radar system connected to and supported by the support and positioned to capture radar data for an area that includes the playing surface; and (c) a local controller stack. The stack includes control circuitry for controlling the radar system and a instructions that will, upon execution, cause a processor to, in response to receiving a command to provide a virtual umpire service: (a) cause the radar system to capture the radar data for the area during a time period; (b) process the radar data to identify, in the radar data, a specified object; (c) determine whether a position of the specified object is within or outside of a boundary of the playing surface; and (d) send a result of the determining to another device of the system.

In some variations of the first embodiment, the processor may include a local processor that is included in the local controller stack.

Some variations of the first embodiment also may include a luminaire attached to the fixture, and at least some of the control circuitry of the local controller stack may also be for controlling the luminaire. If so, then optionally the instructions to send the result of the determining to another device of the system may comprise instructions to cause the luminaire to change one or more characteristics of light output by the luminaire in response to the result.

In some variations of the first embodiment, the instructions to send the result of the determining to another device of the system may comprise instructions to send the result to a scoreboard that is installed proximate to the playing surface for causing the scoreboard to update information displayed in response to the result.

Some variations of the first embodiment also may include a gateway controller that is positioned proximate to the playing surface and that will, in operation, receive data from each of the local controller stacks and send commands to each of the local controller stacks for operation of the radar systems. If so, then optionally: (a) the instructions to send the result of the determining may include instructions to send the radar data to the gateway controller; and (b) the gateway controller may include a memory and programming instructions that will cause the gateway controller to (i) receive first radar data from the radar system, (ii) receive additional radar data from one or more additional radar systems that are installed on one or more additional fixtures positioned proximate the playing surface, and (iii) process the first radar data and the additional radar data to further determine whether the position of the specified object is within or outside of the boundary. In addition, if a luminaire is attached to the fixture, at least some of the control circuitry of the local controller stack also may be for controlling the luminaire.

Some variations of the first embodiment also may include a server that comprises a server processor and programming instructions that will, upon execution, cause the server to, in response to receiving a request from a mobile electronic device: (a) send, to the mobile electronic device for output on a user interface, a user-selectable prompt to trigger operation of the virtual umpire service; and (b) in response to receiving both a user selection of the prompt and payment information for the virtual umpire service from the mobile electronic device, generate the command to operate the virtual umpire service sending the command to the local controller stack. In these variations, the server optionally may further have additional programming instructions to cause the server to, in response to receiving the request: (a) determine whether an administrator is exercising control of the system at the time the request is received; and (b) only generate the command to provide the virtual umpire service after confirming that the administrator is not exercising control of the system at the time the user interface request is received.

In a second embodiment, a method of providing a virtual umpire service includes actions taken a processor of a local controller stack that is connected to a fixture that includes a radar system that is positioned to capture radar data for an area that includes a playing surface. The processor will, in response to receiving a command to provide the virtual umpire service: (a) cause the radar system to capture the radar data for the area during a time period; (b) process the radar data to identify, in the radar data, a specified object; (c) determine whether a position of the specified object is within or outside of a boundary of the playing surface; and (d) send a result of the determining to another device of the system.

In some variations of the second embodiment, the fixture further comprises a luminaire, and the method also comprises, by the processor in response to the command also comprising a command to operate the luminaire, controlling the luminaire while also providing the virtual umpire service. In such variations, sending the result of the determining to another device of the system optionally may comprise causing the luminaire to change one or more characteristics of light output by the luminaire in response to the result.

In some variations of the second embodiment, sending the result of the determining to another device of the system may comprise sending the result to a scoreboard that is installed proximate to the playing surface for causing the scoreboard to update information displayed in response to the result.

In some variations of the second embodiment, a gateway controller may be positioned proximate to the playing surface. The gateway controller may receive data from the local controller stack and one or more additional local controller stacks of additional fixtures that include additional radar systems that are positioned to capture radar data for an area that includes the playing surface. The gateway controller may send commands to each of the local controller stacks for operation of their corresponding radar systems. In addition, sending the result of the determining optionally may comprise sending the radar data to the gateway controller. Further, the gateway controller may, in response to receiving the radar first radar data from the radar system and additional radar data from the additional radar systems, process the first radar data and the additional radar data to further determine whether the position of the specified object is within or outside of the boundary. Optionally, if the fixture also includes a luminaire, then in response to the command also comprising a command to operate the luminaire, the processor may control the luminaire while also providing the virtual umpire service.

In some variations of the second embodiment, in response to receiving a request from a mobile electronic device, a server may (a) send the mobile electronic device, for output on a user interface, a user-selectable prompt to trigger operation of the virtual umpire service and (b) in response to receiving both a user selection of the prompt and payment information for the virtual umpire service from the mobile electronic device, generate the command to operate the virtual umpire service sending the command to the local controller stack. In addition, in response to receiving the request, the server optionally may determine whether an administrator is exercising control of the system at the time the request is received. If so, the server will only generate the command to provide the virtual umpire service after confirming that the administrator is not exercising control of the system at the time the user interface request is received.

In a third embodiment, a computer program product comprises a non-transitory memory device storing programming instructions that will, upon execution, cause a processor of a fixture that includes a radar system, in response to receiving a command to provide a virtual umpire service to perform any of the actions that such a processor may perform in any of the variations of the second embodiment listed above.

In a fourth embodiment, an umpire-as-a-service system includes a radar system and a controller operably connected to the radar system. The controller includes control circuitry for controlling the radar system, along with a memory containing local programming instructions that will, upon execution when the radar system is positioned to capture radar data for a playing surface, cause a processor of the system to, in response to receiving a command to provide a virtual umpire service: (a) cause the radar system to capture the radar data for the area during a time period; (b) process the radar data to identify, in the radar data, a specified object; (c) determine whether a position of the specified object is within or outside of a boundary of the playing surface; and (d) send a result of the determining to another device of the system.

In some variations of the fourth embodiment, additional programming instructions will, upon execution, cause a mobile electronic device to, before the controller receives the command to provide the virtual umpire service, output a user interface that includes a user-selectable prompt to trigger operation of the virtual umpire service. The programming instructions also will cause a processor to, in response to receiving both a user selection of the prompt and payment information for the virtual umpire service via the user interface, generate the command to operate the virtual umpire service and send the command to the controller.

In a fifth embodiment, an umpire-as-a-service system may include a computer program product with programming instructions that will, upon execution, cause a processor to, in response to receiving a service request from a mobile electronic device: (a) identify a radar system that corresponds to the service request, wherein the radar system is positioned to capture radar data for an area that includes a playing surface to which the mobile electronic device is proximate; and (b) cause the mobile electronic device to output a user interface that includes a user-selectable prompt to trigger operation of a virtual umpire service. In response to receiving both a user selection of the prompt and payment information for the virtual umpire service via the user interface: the processor may generate a command to operate the virtual umpire service by causing the radar system to capture radar data for the area that includes the playing surface for the area, and the processor may send the command to a controller that is communicatively connected to the radar system.

Some variations of the fourth and fifth embodiments may incorporate method variants and computer program product variations. In addition, the fourth embodiment, fifth embodiment, and their corresponding method and computer program product variations optionally may incorporate any of the variations of the first, second, or third embodiments.