INTERACTIVE RACQUET SPORT TRAINING SYSTEM

Description

CROSS-REFERENCE TO RELATED APPLICATIONS/PATENTS

This is a non-provisional application in the United States, claiming the benefit of the filing date of the United States provisional application for patent filed on Jun. 27, 2024, assigned Ser. No. 63/664,821 and bearing the title of “INTERACTIVE RACQUET SPORT TRAINING SYSTEM”. The present invention relates to co-owned U.S. Pat. No. 9,275,470, issued on Mar. 1, 2016, to Forkosh et al., and U.S. Pat. No. 11,679,302 issued on Jun. 20, 2023, and entitled ONLINE, REAL-TIME, SYNCHRONIZED TRAINING SYSTEM FOR MULTIPLE PARTICIPANTS. The contents of both U.S. Pat. Nos. 9,275,470 and 11,679,302 are incorporated herein by reference in their entirety.

TECHNICAL FIELD

The present invention relates generally to a system and method for providing online sessions for training, skill developments, and general instruction for racquet sports such as tennis. More specifically, the various embodiments of the invention presented herein relate to a system and method for generating or creating interactive multimedia sessions or programs at a main location and then broadcasting the training sessions to a user device that is remotely located. This process is performed in real-time (as the session is being created), and/or in some embodiments may be at a later time (prerecorded session). The broadcasted interactive session includes a video component to be presented on the user's device, along with a synchronization component used to synchronize the display on the user's device. The interactive sessions include a demonstration mode, wherein a demonstration of a physical activity is streamed in real-time to the participant, followed by a trial period in which the remote participant performs the physical activity, and an analysis application that detects and monitors the motion of the participant or object controlled by the participant weighed against physical requirements of the physical activity to score, rank and report the results of the user and provide behavioral modification feedback to the user.

BACKGROUND

There is a large and growing need for online training of remotely located participants. There are many reasons for this expanding need. One reason is that a specialized and/or well renown athlete or trainer (hereinafter trainer) is greatly limited in the audience that he or she can train, serve or assist. Generally, such athletes or trainers are limited to a physical location, such as a training facility, that is only accessible to local participants. Thus, to expand the trainer's audience, the trainer must rely on participants coming to the trainer's facility or location or, the trainer must travel to other locations. This limitation could be greatly alleviated with an adequate online training system.

The need for online training systems has also recently experienced a drastic increase in view of the global pandemic of Covid-19 that has forced the world to rethink and reevaluate participation in activities that require groups of people to be in close proximity with others or that require mass transient transportation.

Many problems arise in attempting to create and furnish an online training experience that would rival the experience one would get in a “in facility” training system, such as actually being on a tennis court with an instructor. During the Covid-19 pandemic, many have tried hosting such events using technology that was really developed only for conferencing, not physical training. As such, the user experience with online training has at best, been significantly inadequate.

There are many challenges that must be faced in the provision of an online training system and method that is geared towards a remote user or even multiple party audiences that are remotely located. This is especially true when the training requires monitoring and judging the performance of the participant and synchronizing the operational mode of remotely located devices with a live video feed. The challenge is to create a look and feel that gives the participant an experience that is as functional as an “in facility” or “on location” experience and that creates a sense of “being together”.

The present disclosure presents embodiments that address these and other needs in the art.

In the realm of racquet sports, there is a particular need in the art for an online, real-time training system. Rather than having to reserve a court ahead of time to schedule training and practice, such a system can allow a user to train in the comfort of their own home, garage, personal courts, etc. at their own schedule. In addition, such a system can allow a user to have access to training and instruction that normally could not be obtained easily, if at all. Thus, having a system that would give a remote user access to world class talent that normally would never be attainable, could become a reality.

SUMMARY

The above-described needs in the art, as well as other needs, are alleviated through the use of various embodiments of the invention disclosed herein and the technical solutions achieved by providing a system and method for providing a remote interactive training, instructional and practice session to participants in a real-time manner over a network, and monitoring and tracking the performance of the participants.

In general, an exemplary embodiment is a training system that includes an in-studio system and one or more user systems that are interconnected through a network, such as the Internet as a non-limiting example. Together, the systems operate to provide an online training system that includes an instructional portion, a performance portion and a results portion. The training system, referred to as an “in-studio system”, operates as the brains or control center of the various embodiments of the training system. The in-studio system is manned by a trainer or operator that is located in the studio; however, it should be appreciated that in some embodiments the operator may be remotely located from the studio and use a setup similar to the participants and user systems. In some embodiments, the trainer or operator may utilize the in-studio system to create training modules and/or training programs. In other embodiments, the trainer or operator may simply build select training programs from a library of previously created training programs or create training programs from a library of previously created training modules.

The operator also interacts with the in-studio system to launch a training program in which one or more users, utilizing a user system, can participate. A training program consists of one or more training modules. Each training module is configured to first run in a demonstration mode, in which the operator streams a demonstration of the physical activity required by the training module to one or more user systems and/or simply provides instructions and training pertaining to particular skill sets or activities that the operator or trainer wants to conduct with the user. Users can thus view the demonstration of the physical activity on the user systems and receive the instruction. The training module then enters a training or trial mode in which each of the participants then engage in performing the physical activity. The user systems present a video/graphical display to show various performance criteria of the training module. The user systems may also include a sensor mechanisms, such as one or more cameras, motion sensors, cameras that are built-in to the user devices (e.g. iphone/ipad) etc. The motion sensors may be configured to monitor the activity of the user weighed against the performance criteria. Once the trial mode is completed, the data gathered by the user system cameras is then analyzed and evaluated and presented back to the participants. Feedback is calculated and presented in real-time as well as after the session. For example, the system can present the swing speed, reaction time and rep count analytics during the “trial mode” as well as after, as a non-limiting example.

Thus, it can be appreciated that various embodiments of the present invention enable a training program to be simultaneously presented to multiple participants over the Internet. The overall operation of the training system can be controlled by the operator who can initiate the execution of various training modules, control the transition from a demonstration mode to a trial mode and then to a results mode. The operator can maintain control of the system and receive real-time feedback regarding how the participants are doing in the training. Utilizing such embodiments of the training system, the operator can actively engage with the participants, provide input and advice, and help to create the look and feel of an in-facility experience. Further, in non-real-time embodiments, a user has more control over the timing of the training module, having the ability to rewind and redo certain aspects of the training, and then the operator can receive a results report at a later point in time and evaluate the user's progress.

An exemplary embodiment includes a method for providing remote training to one or more participants. The method includes the action of first selecting a particular training program to be executed. The particular training module may reside in the memory of a server or may be created on the fly using a training program editor. The selection of the particular training module may be performed by an operator of the training system or based on a program schedule that a server accesses and then automatically selects the programs according to the schedule. In addition, a user may be able to select from a library of prerecorded sessions. The particular training program may incorporate one or more training modules and each training module defines a plurality of modes or phases. Once the particular training program is selected, the particular training program is then launched, executed or run on the server. Again, running the particular training program can be invoked by the operator or be automated by the server based on the program schedule. While running the particular training program, the training modules within the particular training program are sequentially selected from the training program be to executed. It should be appreciated that once a training module is selected, executed and completed, the next training module in the particular training program may be automatically selected and executed by the server or, the operator may control the timing of selecting and executing the next training module. Further, in some embodiments the server may automatically select and begin the next training module but the operator may have the ability to pause or stop the execution if desired.

While executing the training module, the server transmits video content associated with the selected training module to one or more user devices, wherein each of the one or more user devices is operated by one of the one or more participants. Further, for each phase of the training module, the server embeds data within the video content of the current training module to control the operational mode of the user devices in synchronization with phases or modes of the selected training module. The training system also operates to collect motion data utilizing each of the one or more user devices, wherein the motion data represents a correlation of physical activities of the participant associated with a particular user device and physical actions required by the selected training module. In general, the collected motion data is analyzed and in response, the server creates a results report. While the collection of the data is accomplished with the user devices and the results reports are generated by the server, it should be appreciated that the processing and analyzing of collected data can be performed by the app running on the user devices, the server and/or a combination of both.

In an exemplary embodiment, the user application does all the tracking and analyzing of the motion on the user device and generates metric data. For instance, in a tennis focused system, the metric data can include reaction speed of the user, accuracy of the maneuver, speed of racquet motion, average speed of racquet motion, time to get racquet into hitting position, timing of swing, direction of swing, angle of racquet face, etc. and other metrics relating to the participant's performance as non-limiting examples. More specifically, the user devices perform the motion tracking and data collection at the user locations. The user devices then analyze the collected data. Thus, the user devices determine the number of reps completed by the participant, as well as other metrics. The user devices then display these metrics locally to the participant. The user device may also upload the process data or metrics data to the server over the Internet. In other embodiments, these operations can be shared between the user devices and the server, or some or all of the operations can be performed by the server as well.

Data may be embedded inside the video transmitted to the user devices to instruct or control the user device as to what mode it should be in and associated data with the mode.

Without regard to the particular structure, the motion data is collected, analyzed and then prepared for a results report. The results report can be presented to the user as it is being created or, the user devices may simply hold the information in memory until receiving a command from the server to enter results mode. Once receiving such a command, the user devices present the results data on the displays of the user devices.

The next training module in the training program is then selected and the afore-described process can be automatically repeated for the next training module or may be initiated by operator control.

An exemplary embodiment includes a method for providing interactive training for a racquet sport. The method includes selecting a particular training program residing in the memory of a server, the particular training program comprising one or more training modules that demonstrate a particular skill. The server executed the particular training program in cooperation with a trainer, wherein executing the particular training program comprises the following actions:

• • (a) sequentially selecting a current training module within the particular training program;
  • (b) transmitting to a training window of the user device video content of an instructor demonstrating the particular skill of the training module for a particular time period;
  • (c) subsequent to the particular time period, transmitting to a performance window of a user device video content associated with the particular skill including physical and timing performance targets that a participant is expected to meet by moving a racquet, overlaid with a video feed of the participant performing the particular skill with the racquet;
  • (e) monitoring the motion of the racquet in relationship to the performance targets by collecting motion data that represents the participant performing the particular skill with the racquet;
  • (f) analyzing the motion data to create a results report.

In some embodiments, the physical and timing performance targets represent an expected direction that the racquet is to be moved and the step of monitoring the motion of the racquet in relationship to the performance targets comprises detecting if the racquet moves in the expected direction.

In some embodiments, the action of transmitting to a performance window of a user device video content associated with the particular skill including physical and timing performance targets that a participant is expected to meet by moving a racquet further comprises transmitting an indicia that identifies the expected direction.

In some embodiments, the action of transmitting to a performance window of a user device video content associated with the particular skill including physical and timing performance targets that a participant is expected to meet by moving a racquet further comprises transmitting an indicia that identifies the expected direction and an expected speed.

In some embodiments, the physical and timing performance targets represent an expected timing that the racquet is to be moved through the performance targets and the step of monitoring the motion of the racquet in relationship to the performance targets comprises detecting if the racquet moves through the performance target at the expected timing.

In some embodiments, the action of transmitting to a performance window of a user device video content associated with the particular skill including physical and timing performance targets that a participant is expected to meet by moving a racquet further comprises transmitting an indicia that identifies the expected timing.

In some embodiments, the action of transmitting to a performance window of a user device video content associated with the particular skill including physical and timing performance targets that a participant is expected to meet by moving a racquet further comprises transmitting an indicia that identifies the expected timing by modifying the visual presentation of the indicia such that the visual presentation of the indicia coincides with the expected timing.

In some embodiments, the action of modifying the visual presentation of the indicia such that the visual presentation of the indicia coincides with the expected timing comprises changing the color or the intensity of the indicia.

In some embodiments, the indicia simulates a moving object that is to be hit by the racquet and the action of modifying the visual presentation of the indicia such that the visual presentation of the indicia coincides with the expected timing comprises moving the object across the display into a target zone that represents the location that the racquet is to hit the object.

In some embodiments, the racquet includes a receiver and a haptic feedback generator and the server transmits a signal to the racquet to provide haptic feedback of the racquet striking or missing the object.

In some embodiments, the racquet includes visual site located on the face of the racquet, and the step of monitoring the motion of the racquet in relationship to the performance targets comprises detecting the visual site of the racquet.

Another embodiment includes system for remote training. The system includes a server having a processor and a memory element communicatively coupled to the processor, the memory element including instructions defining a server application. Further, the system includes a user device communicatively coupled to a camera, a display device and a speaker, further, the user device including a processor and a memory element, the memory element of the user device including instructions defining a user device application.

In such embodiments, when executed on the server, the server application is configured to launch a particular training program. In response to launching the particular training program, the server application is configured to:

• • (a) sequentially select a current training module from the particular training program;
  • (b) transmit to a training window of the user device video content of an instructor demonstrating the particular skill of the training module for a particular time period;
  • (c) subsequent to the particular time period, transmitting to a performance window of a user device video content associated with the particular skill including physical and timing performance targets that a participant is expected to meet by moving a racquet, overlaid with a video feed of the participant performing the particular skill with the racquet;
  • (e) monitoring the motion of the racquet in relationship to the performance targets by collecting motion data that represents the participant performing the particular skill with the racquet;
  • (f) analyzing the motion data to create a results report.

In some embodiments, the physical and timing performance targets represent an expected direction that the racquet is to be moved and the system monitors the motion of the racquet in relationship to the performance targets by detecting if the racquet moves in the expected direction.

In some embodiments, the physical and timing performance targets further comprise an indicia that identifies the expected direction of the racquet.

In some embodiments, the indicia identifies the expected direction and an expected speed of the racquet.

In some embodiments, the physical and timing performance targets further comprise an indicia that identifies an expected timing for moving the racquet.

In some embodiments, the indicia that identifies the expected timing is modified by the system to provide a visual presentation of the indicia such that the visual presentation of the indicia coincides with the expected timing.

In some embodiments, the system modifies the visual presentation of the indicia such that the visual presentation of the indicia coincides with the expected timing by changing the color or the intensity of the indicia.

In some embodiments, the system simulates a moving object that is to be hit by the racquet and moves the object across the display into a target zone that represents the location that the racquet is to hit the object, and wherein the racquet includes a receiver and a haptic feedback generator and the system transmits a signal to the racquet to provide haptic feedback if the racquet hits or misses the object in the target zone.

In some embodiments, the racquet includes visual site located on the face of the racquet, and the system monitors the motion of the racquet in relationship to the performance targets by detecting the visual site of the racquet.

These embodiments and other embodiments, features, aspects and functionalities are more specifically described in the detailed description of the various drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be more fully understood from the detailed description of exemplary embodiments considered in conjunction with the following drawings, of which:

FIG. 1 is a functional block diagram of the components of an exemplary embodiment of system, sub-system, or platform operating as a controller or processor that could be used in various embodiments of the disclosure for controlling aspects of the various embodiments or for implementing all or portions of the disclosed embodiments.

FIG. 2 is a conceptual system diagram on an exemplary embodiment of the training system for use in a racquet sport such as tennis.

FIG. 3 is a conceptual illustration of an exemplary training module including a series of screens that can be created by an operator and instructing a participant to conduct a series of tennis shots including a forehand and backhand sequence.

FIG. 4A illustrates a photograph, screen shots, or still image of the video feed of an actual implementation of the onsite component.

FIG. 4B illustrates a photograph, screen shots, or still image of the video feed of an actual implementation of the onsite component.

FIG. 4C illustrates a photograph, screen shots, or still image of the video feed of an actual implementation of the onsite component.

FIG. 5A illustrates a photograph, screen shot, or still image of the video feed of an actual implementation of the onsite component requesting the participant to hit a forehand shot.

FIG. 5B illustrates a photograph, screen shot, or still image of the video feed of an actual implementation of the onsite component requesting the participant to hit a forehand shot.

FIG. 5C illustrates a photograph, screen shot, or still image of the video feed of an actual implementation of the onsite component requesting the participant to hit a forehand shot.

FIG. 6A is a conceptual design for presenting the flat serve skill training within a training module.

FIG. 6B is a conceptual design for presenting the slice serve skill training within a training module.

FIG. 6C is a conceptual design for presenting the kick serve skill training within a training module.

FIG. 6D is a conceptual design for presenting the forehand flat shot skill training within a training module.

FIG. 6E is a conceptual design for presenting the forehand topspin shot skill training within a training module.

FIG. 6F is a conceptual design for presenting the forehand slice shot skill training within a training module.

FIG. 6G is a conceptual design for presenting the forehand volley flat shot skill training within a training module.

FIG. 6H is a conceptual design for presenting the forehand volley topspin shot skill training within a training module.

FIG. 6I is a conceptual design for presenting the forehand volley slice shot skill training within a training module.

FIG. 6J is a conceptual design for presenting the backhand flat shot skill training within a training module.

FIG. 6K is a conceptual design for presenting the backhand topspin shot skill training within a training module.

FIG. 6L is a conceptual design for presenting the backhand slice shot skill training within a training module.

FIG. 6M is a conceptual design for presenting the backhand volley flat shot skill training within a training module.

FIG. 6N is a conceptual design for presenting the backhand volley topspin shot skill training within a training module.

FIG. 6O is a conceptual design for presenting the backhand volley slice shot skill training within a training module.

FIG. 6P is a conceptual design for presenting the forehand moon ball or lob shot skill training within a training module.

FIG. 7 is a still frame taken from a live racquet training system in operation.

FIG. 8 illustrates a flow diagram of screens for an exemplary implementation of an intensity-based target for showing the participate timing of the swing.

FIG. 9 is a timing diagram illustrating an exemplary implementation of using intensity on the target along with a moving ball to show the participant timing of the swing.

FIG. 10 is a conceptual illustration of a training program editor that could be employed in various embodiments of the training system.

FIG. 11 is a flow diagram illustrating the operation of the system after launching of a training program.

FIG. 12 is a signal diagram representative of the use of embedded data commands to provide synchronization of the user devices with the video content and server application for various embodiments.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

The present invention, as well as features and aspects thereof, is directed towards a synchronized training system for racquet-based sports in which an operator can present a training program to multiple participants, the participants can respond by making required athletic movements with their body and racquet, and the training system can monitor and report the performance of the participants.

In the description and claims of the present application, each of the verbs, “comprise”, “include” and “have”, and conjugates thereof, are used to indicate that the object or objects of the verb are not necessarily a complete listing of members, components, elements, or parts of the subject or subjects of the verb.

In this application the words “unit”, “application”, “program” and “module” are used interchangeably. Anything designated as one of these items may be a stand-alone element or incorporated with other elements. Each such items may be any one of, or any combination of, software, hardware, and/or firmware.

In general, an exemplary embodiment of the training system includes an onsite component that a participant utilizes and an in-studio component that is used to feed, control and monitor the onsite component. The onsite component includes a video display and an audio output from which the participant receives instructions, commands, challenges, etc. Further the onsite component also includes one or more sensory inputs that are used to monitor and/or receive information regarding the user's reactions to various instructions, commands, challenges that are presented to the participant. The sensory inputs may include a video camera that can monitor and decipher movements of the participant and the racquet. In such embodiments, the user may wear one or more passive sensory tags or targets, such as QR codes, bar codes or some other camara readable code. For instance, the sensory tags can be mounted to the participant's wrist by using sweat bands or bracelets, to the participant's head by being embedded in a headband or front of a hat, to the participant's belt, upper thighs, knees, ankles, etc. Further, the sensor tags may also be mounted to the participant's racquet, such as on the face of the strings, the top of the racquet, the handle, etc.

Some embodiments may also include an audio mic for recording sounds from the participant.

Other embodiments may use active sensory tags in connection with more passive sensory inputs. For instance, the sensory tags may include active motion sensors such as accelerometers and/or gyroscopes. In such embodiments, the sensory tags can still be mounted on the participant and racquet as previously described, and as the participant moves, signals can be transmitted to and received by the sensory inputs of the onsite component.

Some embodiments may use a combination of active sensory tags and passive sensory tags.

In general, various embodiments of the training system may be configured to operate within a distributed system. The distributed system may include an in-studio system and one or more user systems that are interconnected through a network, such as the Internet. Together, the systems operate to provide an online and interactive training system that includes an instructional/training portion, a performance portion, and a results portion. The in-studio system is the brains or control center of the various embodiments of the training system and is manned by a trainer or operator. The trainer or operator may utilize the in-studio system to create training modules and/or training programs. Within this disclosure, the term training module is a training unit that may focus on a certain physical activity, exercise, motor skill, etc. and can be operated in a demonstration mode, training mode, results mode and leaderboard mode as a non-limiting example. Further, within this disclosure, the term training program is a class or training session that includes one or more training modules.

The operator also interacts with the in-studio system to launch a training program in which one or more users, utilizing a user system, can participate. A training program may consist of one or more training modules. Each training module may be configured to first run in a demonstration mode, in which the operator streams a demonstration of the physical activity required by the training module to one or more user systems. Users can thus view the demonstration of the physical activity on the user systems. The training module may then enter a training or trial mode in which each of the participants then engage in performing the physical activity. The user systems present a video/graphical display to show various performance criteria of the training module. The user systems also include a sensory input sub-system that is configured to monitor the activity of the user. The results of the monitored activity can then be weighed against the performance criteria. Once the trial mode is completed, the training system may switch to a results mode. In the results mode, the monitored activity of the user may be presented on the user devices. Upon receiving the command to enter results mode, the user devices may then commence analyzing data collected while monitoring the activity to generate a results report or, the user devices may analyze the data while it is being collected and simply present the results report upon receipt of the command to enter results mode. In addition, in some embodiments the analyzed and evaluated data may be presented or transmitted to the in-studio server as well. Finally, the training system can enter a leaderboard mode in which the results of all participants can be ordered or ranked and presented back to the in-studio server and/or participants.

During operation, the in-studio system continuously transmits a video feed to the user systems. This video feed may be constructed of different video content for different operating modes. For instance, during the demonstration mode, the video feed may include a live or pre-recorded video performance of the instructor demonstrating the physical activity. The video feed may also include video that provides instructions or guidance to assist the users in the performance of the physical activities, as well as performance targets that the users are to attempt to meet. The video feed may also include results data and/or leaderboard standings based on data collected from the user systems. Other embodiments may also include other video content as well.

As previously mentioned, a technical hurdle in such a system is ensuring that the user systems are synchronized with the video feed. For example, when the training system switches from demonstration mode to trial mode, the user systems must enter the trial mode to start monitoring the motion of the user and/or object while displaying the instructional or guidance video. Due to the fact that such video content can be buffered, the actual time of receipt may vary between the user systems. To ensure that the user systems are synchronized with the video feed, data commands or controls are embedded into the video feed to control the state or mode of the user systems.

With this general understanding of the overall operation of the various embodiments of the training system, specific aspects, features, functionality and operations of the various embodiments are presented in conjunction with the figures.

FIG. 1 is a functional block diagram of the components of an exemplary embodiment of system, sub-system, or platform operating as a controller or processor 100 that could be used in various embodiments of the disclosure for controlling aspects of the various embodiments or for implementing all or portions of the disclosed embodiments. As such, the in-studio system, the user systems and/or components of each could be implemented on such an exemplary platform. It will be appreciated that not all of the components illustrated in FIG. 1 are required in all embodiments or all components of a training system but, each of the components are presented and described in conjunction with FIG. 1 to provide a complete and overall understanding of an exemplary platform that can be utilized for implementing multiple systems.

The controller can include a general computing platform 100 illustrated as including a processor/memory device 102/104 that may be integrated with each other or, communicatively connected over a bus or similar interface 106. The processor 102 can be a variety of processor types including microprocessors, micro-controllers, programmable arrays, custom IC's etc. and may also include single or multiple processors with or without accelerators or the like. The memory element of 104 may include a variety of structures, including but not limited to RAM, ROM, magnetic media, optical media, bubble memory, FLASH memory, EPROM, EEPROM, computer-on-chip, etc. The processor 102, or other components in the controller may also provide components such as a real-time clock, analog to digital convertors, digital to analog convertors, etc. The processor 102 also interfaces to a variety of elements including a control interface 112, a display adapter 108, an audio adapter 110, and network/device interface 114. The control interface 112 provides an interface to external controls, such as sensors, actuators, drawing heads, nozzles, cartridges, pressure actuators, leading mechanism, drums, step motors, a keyboard, a mouse, a pin pad, an audio activated device, as well as a variety of the many other available input and output devices or, another computer or processing device or the like. The display adapter 108 can be used to drive a variety of alert elements 116, such as display devices including an LED display, LCD display, one or more LEDs or other display devices. The audio adapter 110 interfaces to and drives another alert element 118, such as a speaker or speaker system, buzzer, bell, etc. and/or an audio source detector such as a microphone. The network/interface 114 may interface to a network 120 which may be any type of network including, but not limited to the Internet, a global network, a wide area network, a local area network, a wired network, a wireless network or any other network type including hybrids. Through the network 120, or even directly, the controller 100 can interface to other devices or computing platforms such as one or more servers 122 and/or third party systems 124. A battery or power source provides power for the controller 100.

FIG. 2 is a conceptual system diagram on an exemplary embodiment of the training system for use in a racquet sport such as tennis. The illustrated exemplary embodiment includes the afore-described systems or operating centers: (a) in-studio system 200 and (b) user system 250. Although only one user system 250 is illustrated in the diagram, it should be appreciated that any number of user systems can be connected to and operate in conjunction with the in-studio system 200. As such, multiple participants may be engaged in any particular training program.

In an exemplary embodiment, the operation of the in-studio system 200 may allow or enable an operator, instructor or trainer (hereinafter referred to as the “operator”) 210 to create or select already created training modules to be included in a training program. The training modules, as non-limiting examples, may focus on instructing participants to engage in certain physical activities (i.e., flat serve, slice serve, kick serve, forehand flat, forehand topspin, forehand slice, moon ball, backhand flat shot, backhand topspin, backhand slice, approach shot, forehand volley, backhand volley, overhead smash, lob shot, dropshot). The in-studio system 200 may also enable the operator to create or select an already created training program. The training program typically incorporates a single training module or a series of training modules. As such, a training program can be created to include various physical activities or exercises that are geared towards accomplishing specific goals for the participants. As a non-limiting example, a training program can be geared towards training or improving the various skills of a tennis player or other athletes. Once the operator 210 creates a training program, or selects a previously created training program, the operator 210 can invoke the in-studio system 200 to launch the training program.

Once the training program is launched, for each training module in the training program, the in-studio system 200 first enters a demonstration mode in which the operator 210 performs the physical activity presented in, or focused on, in the training module and may provide additional physical and audible instructions related to the physical activity. The operator's 210 actions are recorded or received by an onsite camera along with any audio instructions that the operator 210 may speak or other audio content, and the video and audio content are streamed to one or more user systems 250.

The user system 250 receives and displays the streamed demonstration video and/or audio on a display device or screen 254 and then, the in-studio system 200 and the user system 250 enters the trial mode. In the trial mode, the user system 250 renders a training guide or visual instruction on how to perform the physical activity required in the training module. The user system 250 also operates to monitor, measure and track the performance of the participant.

Once the trial mode is complete for a specific training module, the in-studio system 200 and the user system 250 switch to a results mode. In the results mode, data collected by the user system 250 may be reported back to the in-studio system 200 which cab collects, analyzes and then scores or grades the performance of the participants. This information is then displayed on a display device 214 for the operator 210 and/or reported back to the user system 250 to be displayed to the user on the screen 254.

Once a specific training module is complete, the in-studio system 200 then proceeds to the next training module in the training program and repeats the process.

Looking more closely at FIG. 2, the in-studio system 200 is illustrated as including a recorder 202, a camera 204, an actuator 206 and a server 208. The server 208 includes a display device 214 and a server application 216. An operator 210 can interface to the server 208 using an input device such as a keyboard, mouse-pad, touch screen, voice commands, the actuator 206 or some other input device. While the server 208 is illustrated as including a display device 214, it should be appreciated that the display device 214 may be integrated with the server 208 or be communicatively coupled to the server 208 either using wired or wireless technology. The server 208, may be a platform as illustrated in FIG. 1. The server application 216 is stored within the memory of the server 208 or may be accessible from external memory, the cloud, etc. In operation, the server application 216 is launched by the operator 210 and presents an operator interface to the operator 210. In an exemplary embodiment, the operator interface may include selectable functions for the operator 210, such as, but not limited to, a program editor and a program executor.

The in-studio system 200 can be accessed by one or more user systems 250 through a network 280. The network 280 may typically be the Internet or a global network but, it is also anticipated that the network 280 can also be a wireless network such as WiFi or Bluetooth, or a wired network or cable, any of the aforementioned being utilized for a more localized deployment of the training system.

The user system 250 is illustrated as including a user device 252. The user device 252 may be a laptop computer, a desk computer, a smart phone, a tablet, a notebook etc. The user device 252 is illustrated as including a screen 254, a sensory input 256 and an audio input (i.e., a microphone as a non-limiting example) 258. Further, the user system 250 includes a user device application 260 to create the functionality of the user system 250. The user device application 260 may be internal to the user device 252 or may be accessed from external memory or the cloud. Each user system 250 can connect to and communicate with the in-studio system 200 through network 280 or through other means as previously described. A participant 262 with an actual object 264 (such as a tennis racquet) utilizes the user device 252 to participate in a training program.

Although the in-studio system 200 is described as being co-located, it should be appreciated that in some embodiments, the operator 210 may be remotely located from the server. In such an embodiment, the operator 210 may include a similar system as the user systems 250 and be connected to the server 208 through the network 280. Further, the operator 210 would then have a local recorder 202, camera 204 and optionally an actuator 206 and a computing device or communication system that allows each of these components to be communicatively coupled to the server 208. As a non-limiting example, the operator 210 may include a setup or system quite similar to the user system 250, with the exception that the user application that runs on the operator's 210 system would be able to control and operate the server application 216. As an example, the user application 260 may include an operator 210 login function and a participant login function, wherein if an operator logs into the user application, the operator will have additional controls or capabilities not available to the participant.

It should also be appreciated that for embodiments in which the operator 210 includes an operator system that is remote from the server 208, that multiple operators 210 may simultaneously utilize the services of the server 208 to run different training programs and the server 208 can keep track of which operator 210 system is associated with a particular active training program and which participants or user systems are also engaged in a particular training program.

Server Application. The server application 216 provides certain functionality of the in-studio server system 200 when loaded and executed by a processor. The operator 210 interacts with the server application 216 through an operator interface. As previously mentioned, the operator interface may include at least two distinct elements or functions: (1) an editor and (2) a program executor. In embodiments that include the editor, the server application 216 may enable the operator 210 to interact with a training module editor, a training program editor, and a training program previewer. It should be appreciated that not all of these functions are required in any particular embodiment of the training system but may be included individually or all together in any embodiment of the training system. The program executor enables the operator 210 to launch and present the training program to the participants.

Training Module Editor. An exemplary training module editor may present options for an operator 210 to create a training module from scratch or to modify an existing training module or training module template. A training module can vary in complexity. As a non-limiting example, a training module may simply be a single exercise or physical action that is being required of a participant, trainee, or user (hereinafter referred to as the “participant”). As another non-limiting example, the training module may be a series of exercises or physical actions that can be used to test, train or develop certain motor skills or simply to instruct or guide the participant. Throughout this description, specific examples of applications of the various embodiments will be utilized to further describe the operation of the embodiments. These specific applications should be viewed as non-limiting examples of how embodiments of the invention can be utilized rather than as limitations on the scope of the invention, although certain aspects of the specific applications may also be considered to be patentable inventions or distinguishing limitations for particular embodiments.

One non-limiting example of a specific application is for training participants in the development of racquet handling motor skills for the sport of tennis. As such, a training module in this non-limiting example may simply be focused on one aspect of swinging the racquet, such as a forehand volley, slice serve, or any of the other aforementioned tennis shots. More complex training modules may focus on certain styles of swinging the racquet and body movement. In these listed examples, certain movements by the participants can be defined and anticipated. In creating the training modules, the operator 210 can thus create a graphical presentation illustrating the required physical activity.

FIG. 3 is a conceptual illustration of an exemplary training module including a series of screens that can be created by an operator 210 and instructing a participant to conduct a series of tennis shots including a forehand and backhand sequence. The operator 210 can create the shot series training module utilizing the training module editor. In the illustrated embodiment, the shot series training module is shown as including a series of screens that are progressed through to facilitate training the physical activity of the shot series. Initially SCREEN A is selected to be displayed which illustrates that the participant (shown as a ghosted FIG. 302) is instructed to hit a forehand shot (this series is created for a right-handed player). This is accomplished by placing a green target 312 to the right-handed side of the ghosted FIG. 302 with three arrows point upwards and towards the ghosted FIG. 302 at an angle. The forehand target 312 can be displayed for a duration of X1, a parameter that may be selected by the operator and potentially augmented by the participant. In some embodiments, the targets may simply be one color or black but in other embodiments, the targets may be color coded to help the participant identify the type of shot being requested more readily. The participant is thus instructed to move the tennis racquet being held in the participant's right hand, to the target 312 and then move it in the direction of the arrows 314.

The training module then transitions to SCREEN B to instruct the participant to perform a backhand shot. This is accomplished by placing a pink target 316 to the left-handed side of the ghosted FIG. 302 with three arrows 318 pointing upwards and towards the ghosted FIG. 302 at an angle. The backhand target 316 can be displayed for a duration of X2, a parameter that may be selected by the operator and potentially augmented by the participant. The participant is thus instructed to move the tennis racquet being held in the participant's right hand (or both hands for a two handed backhand), to the target 316 and then move it in the direction of the arrows 318.

In creating, defining or modifying the training module, the operator 210 can select a number of repetitions (i.e. 1 to N) for the training module to cycle through, a time limit to run the training module, set the module to run indefinitely until either the operator or participant request a transition, or set the module to run until the participant meets certain performance goals. The operator 210 may also define other parameters for the physical activity, such as but not limited to the tolerance of how close the participant follows the instructions, the speed of transitions, the height of the targets (which may be based on the height of the participant and other parameters as will is presented in more detail in the following descriptions of embodiments and features.

Rather than transitioning to different screens as illustrated, the training module may be more of a live/active video in which the illustrated targets actually move on the screen as a video rather than switching from screen to screen. Further, in some embodiments, each of the target positions may be illustrated at the same time and the current desired or active target may be highlighted in some fashion (i.e., flashing or being brighter). In yet other embodiments, each of the target positions may be illustrated at the same time and as the racquet utilized by the user moves, the target on the screen changes to show the current position racquet.

In some embodiments, the training module is presented as an overlay of a video of the participant. As such, the video of the participant and the racquet, in the illustrated example, are presented on the screen 254, and the target, motion indicators, etc., are overlaid on the participant video thus allowing the participant to visually gauge his or her performance of the physical activity.

In each of these embodiments, an element of the training module is a simulated object that correlates to an actual object that is utilized by a user. A goal of the training module is to demonstrate or guide the user in the required physical activity and then monitor the actions of the user to determine how close the user's performance tracks with the simulated physical activity in the training module. The object that is being utilized by the user of the user system 250 is configured to be tracked by the user device 252. This can be accomplished in a variety of manners. In one embodiment, the actual object utilized by the user may have a special design, color or indicia that can be easily tracked by the camera 256 and user application 260 of the user device 252. In other embodiments, the actual object may include other technologies such as accelerometers, GPS receivers, RFIDs, gyroscopes etc. that work in communication with the user device 252 to track the motion, position and location of the actual object.

The training module editor may enable the operator 210 to create a variety of physical activities, such as the illustrated shot series in FIG. 3, and then allow the operator 210 to save the training module on the server 208, such as in a library of training modules.

Further details of the studio system are presented below simply to convey an enabling embodiment of a platform that is suited for embodiments of the present invention. But before that, the actual details of the operation of the racquet sport training embodiment are described, along with features, aspects and advantages of the embodiment.

FIG. 4A, FIG. 4B and FIG. 4C illustrate photographs, screen shots, or still images of the video feed of an actual implementation of the onsite component. Looking at FIG. 4A, the display 400 presented to the participant 412 includes two components: (a) the instructor section 402 and (b) the participant section 404. The instructor section 402 includes a display window where the instructor 414 can demonstrate moves and provide instruction to the participant. During live sessions, the instructor 414 may also provide live feedback to the participant 412 with regards to the participant's 412 performance. In the illustrated example, the participant 412 is being instructed to hit a back hand (please note this is a left-handed participant). The target 420 is presented on the screen in window 404, which also includes a live presentation of the participant 412. The participant 412 is holding racquet 422. The user device 250, in conjunction with the sensory input 256, detects the position of the racquet 422 and displays racquet position 424 on the screen in the participant section 404. Looking at FIG. 4B, the participant 412 has moved the racquet into the vicinity of the target 420 and the system has moved the racquet position identifier 424 accordingly. As the participant 412 swings through the shot, the participant moves the racquet in the directions of the arrows 421. FIG. 4C illustrates the participant 412 having made the backhand swing. The arrows 421 are moving in the direction of the participant's swing and the position identifier 424 is over the participant's left shoulder along with the racquet 422.

FIG. 5A, FIG. 5B and FIG. 5C illustrate photographs, screen shots, or still images of the video feed of an actual implementation of the onsite component requesting the participant 412 to hit a forehand shot (please note that this is a left-handed participant). Looking at FIG. 5A, the target 520 is presented on the screen in window 404, which also includes a live presentation of the participant 412. The participant 412 is holding racquet 422. The user device 250, in conjunction with the sensory input 256, detects the position of the racquet 422 and displays racquet position 424 on the screen in the participant section 404. Looking at FIG. 5B, the participant 412 has moved the racquet into the vicinity of the target 520 and the system has moved the racquet position identifier 424 accordingly. As the participant 412 swings through the shot, the participant moves the racquet in the directions of the arrows 521. FIG. 4C illustrates the participant 412 having made the forehand swing. The arrows 421 move in the direction of the participant's swing and the position identifier 424 is over the participant's left shoulder along with the racquet 422.

The various embodiments of the racquet sport training system can be at various levels of sophistication. For instance, an embodiment may be limited to just working the forehand and backhand shots. While other embodiments may include any or all of the typical tennis shots, including but not limited to the following shots: flat serve, slice serve, kick serve, forehand flat, forehand topspin, forehand slice, moon ball, backhand flat shot, backhand topspin, backhand slice, approach shot, forehand volley, backhand volley, overhead smash, lob shot, and dropshot. Each of these shots can also be uniquely represented on the display of the onsite system or user device 250.

These shots can be categorized as follows:

• • Serves: flat serve, slice serve, kick serve
  • Forehand Shots: forehand flat, forehand topspin, forehand slice, forehand volley,
  • Backhand Shots: backhand flat, backhand topspin, backhand slice, backhand volley
  • Miscellaneous Shots: moon ball, approach, overhead smash, lob, and drop

In an exemplary embodiment, all or some of these shots may be included such that a training module may present exercises to the participant for various types of shots. One non-limiting example of implementing these shots into a racquet sport training is provided below

Serves

Flat Serve. The flat serve is one that most beginners learn at the onset but it is also a shot that is used at many levels of tennis expertise. The flat serve delivers the ball with minimal amounts of spin and thus, it can attain high speeds and low bounce. This makes the flat serve difficult to return. Because of the speed and power of the flat serve, it can be challenging to deliver the serve with accuracy. FIG. 6A is a conceptual design for presenting the flat serve skill training within a training module. In the illustrated screen 6A, the participant 602 sees the display of a target 610 with direction or path movement arrows 630. In this case, the arrows 630 indicate that the participant 602 should swing substantially flat across the front of his or her body. As skills are developed by the participant 602, the position of the flat serve can be gradually moved upward until it is in the overhead position.

Slice Serve. The slice serve is a power weapon in the arsenal of the tennis player and can easily confuse or surprise the opponent. The slice serve is executed by throwing the ball overhead and then hitting the outer edge of the ball on the swing of the racquet such that the face of the racquet strings slides across the outer edge of the ball and applies side spin. This side spin can adjust the ball's bounce on the serve, making it difficult for the opponent to anticipate. The effectiveness of the slice serve varies by playing surface, with it being most powerful on fast surfaces such as grass. FIG. 6B is a conceptual design for presenting the slice serve skill training within a training module. In the illustrated screen 6B, the participant 602 sees the display of a target 611 in the overhead position. The target includes direction arrows 631 to assist the participant 602 in knowing what the swing path or type is. In this example, the arrows point outward and are positioned closer together to indicate that the participant 602 needs to move the face of the racquet along the outside edge of the ball to create the spin necessary for achieving the slice serve.

Kick Serve. The kick server is a difficult tennis maneuver to master but it is also a very powerful tool in competition. The goal of the kick serve is to generate a significant amount of topspin on the ball during an overhead serving which. This is difficult in that the participant 602 needs to position the racquet such that he or she can slide the face of the racquet over the top edge of the ball. This generates topspin on the ball, which produces a high bounce when delivered within the opponent's court. This makes it great for pushing opponents back and is a good choice of serve for opponents that struggle returning high balls. The kick serve, while generating a high bounce also loses some of the power that can be attained with the flat serve. FIG. 6C is a conceptual design for presenting the kick serve skill training within a training module. In the illustrated screen 6C, the participant 602 sees the display of a target 612 in the overhead position. The target includes direction arrows 632 to assist the participant 602 in knowing what the swing path or type is. In this example, the arrows point upward and are positioned closer together to indicate that the participant 602 needs to move the face of the racquet along the topside edge of the ball to create the topspin necessary for achieving the kick serve.

Another key aspect of serves is ensuring that the participant does not cross the service line prior or move from the service side half court to the receiving side half court prior to striking the ball with the racquet. As such, FIG. 6C further includes a service line indicator 660 and a court side divider line 662. In such an embodiment, the sensory input 256 can cross-correlate the timing of the participant 602 striking the ball along with the position of the participant 602 relative to the lines 660 and 662. As such, the system can notify the participant 602 if the serve was valid or illegal.

Yet another key aspect of serving is having a good ball toss. At the beginning of a serve, the player tosses the ball into the air, typically above the players head, and then as the ball descends, the player strikes the ball with the racquet to serve the ball to the other side of the net and into the service court of the opponent. An aspect that can be included in one or more embodiments is simulating the serve ball toss. In such embodiments, the system can simulate the ball being tossed and present the trajectory of the ball on the screen to allow the participant to work on his or her timing.

Simulating a ball on the screen is further described herein below. Another aspect that can be incorporated into various embodiments is to allow the participant to use a live ball in practicing the toss and service. In such embodiments, the ball can include a target to enable the sensory input 256 to track the movement and trajectory of the ball, or other motion tracking elements as described herein, and equivalents thereof can be embedded or affixed to the ball. As such, the system can calculate, track and monitor the trajectory of the ball as well as track and monitor the participant swinging the racquet. In each of these embodiments, the system can provide feedback to the participant regarding the toss (i.e., it was too shallow, too high, too forward, too backward, too left, too right, etc.), the timing of the swing, the direction of the swing, the amount of spin that may have been applied to the ball, etc.

Forehand Shots

Forehand shots are played from the participant's dominant side. For instance, a right-handed player will strike forehand shots on the players right side using his or her right hand to hold the racquet.

Forehand Flat. In the forehand flat shot, the player strikes the ball with a direct, horizontal swing and as such, the shot does not generate much spin on the ball. Because there is a minimal amount of spin, the ball does not bounce very high in the opponent's court. FIG. 6D is a conceptual design for presenting the forehand flat shot skill training within a training module. In the illustrated screen 6D, the participant 602 sees the display of a target 613 with direction or path movement arrows 633. In this case, the arrows 633 indicate that the participant 602 should swing substantially flat across the front of his or her body.

Forehand Topspin. A topspin forehand is a very common shot in tennis and one of the first that many beginners learn. The player generates topspin by allowing the racket to brush the top of the ball during the swing. As such the racquet is moved forward and upward with a slight forward lean on the face of the racquet. The ball rotates towards your opponent and thus, when it strikes the ground in the opponent's court, the ball will have a high and forward bounce. In addition, the forehand topspin shot causes the ball to arc over the net giving the ball striker more net clearance. FIG. 6E is a conceptual design for presenting the forehand topspin shot skill training within a training module. In the illustrated screen 6E, the participant 602 sees the display of a target 614 with direction or path movement arrows 634. In this case, the arrows 634 indicate that the participant 602 should swing forward and upward across the front of his or her body.

Forehand Slice. The forehand slice can be viewed as the opposite of a forehand topspin shot. To execute a forehand slice, the player causes the racket to brush the bottom edge of the ball as the ball is struck. This causes the ball to rotate backward, away from your opponent. The result of the forehand slice is that the ball is slowed down and when it hits the court on the opponent's side, it tends to have a low bounce. FIG. 6F is a conceptual design for presenting the forehand slice shot skill training within a training module. In the illustrated screen 6F, the participant 602 sees the display of a target 615 about waist high. The target includes direction arrows 633 to assist the participant 602 in knowing what the swing path or type is. In this example, the arrows are pointing downward and are positioned closer together to indicate that the participant 602 needs to move the face of the racquet along the bottom edge of the ball to create the spin necessary for achieving the slice serve.

Forehand Volley. The forehand volley is very similar to any of the previously described forehand shots with the exception that the shot is made when the player is near the front of the net and the shot is made before the ball bounces in the player's court. This is one of the most common shots to play when a player is up at the net and can hit the ball before it bounces. The backhand volley also is more of a punch than a swing as the ball's momentum is used rather than the racquet momentum. FIG. 6G, FIG. 6H and FIG. 6I can be compared to FIG. 6D, FIG. 6E and FIG. 6F as the shots are the same with the exception that the ball is struck before it bounces. As such, the targets 616, 617 and 618 respectively are illustrated with dotted lines to indicate that these shots are taken at the net and before the ball bounces and the racquet directions are respectively indicated by arrows 636, 637 and 638. Further, the arrows are shown as including only one or two arrows to indicate that this is a punch shot rather than a full swing.

Backhand Shots

Backhand shots are more difficult than forehand shots in that they are hit cross-body on the player's non-dominant side. Backhand shots can be played with either one hand or two hands. As a generalization, one-handed backhands can generate more power and spin, whereas two-handed backhands can help give more control.

Backhand Flat shot. Similarly to the forehand flat, the backhand flat shot doesn't add a substantial amount of spin. The shot is a quick return with low bounce requiring the opponent to respond quickly. FIG. 6J is a conceptual design for presenting the backhand flat shot skill training within a training module. In the illustrated screen 6J, the participant 602 sees the display of a target 619 with direction or path movement arrows 639. In this case, the arrows 639 indicate that the participant 602 should swing substantially flat across the front of his or her body.

Backhand Topspin. The backhand topspin is slightly different from the forehand, as the racquet must be swung up on the ball to get it to rotate. This can be a difficult maneuver from the non-dominant side. FIG. 6K is a conceptual design for presenting the backhand topspin shot skill training within a training module. In the illustrated screen 6K, the participant 602 sees the display of a target 620 with direction or path movement arrows 640. In this case, the arrows 640 indicate that the participant 602 should swing forward and upward across the front of his or her body.

Backhand Slice. The backhand slice is typically an easier shot to deliver from the non-dominant side. Similarly to the forehand slice, the backhand slice slows the ball down and results in little bounce. FIG. 6L is a conceptual design for presenting the backhand slice shot skill training within a training module. In the illustrated screen 6L, the participant 602 sees the display of a target 621 about waist high. The target includes direction arrows 641 to assist the participant 602 in knowing what the swing path or type is. In this example, the arrows are pointing downward and are positioned closer together to indicate that the participant 602 needs to move the face of the racquet along the bottom edge of the ball to create the spin necessary for achieving the slice serve.

Backhand Volley. The backhand volley is very similar to any of the previously described backhand shots with the exception that the shot is made when the player is near the front of the net and the shot is made before the ball bounces in the player's court. The backhand volley also is more of a punch than a swing as the ball's momentum is used rather than the racquet momentum. FIG. 6M, FIG. 6N and FIG. 6O can be compared to FIG. 6J FIG. 6K and FIG. 6L as the shots are the same with the exception that the ball is struck before it bounces. As such, the targets 622, 623 and 624 respectively are illustrated with dotted lines to indicate that these shots are taken at the net and before the ball bounces and the racquet directions are respectively indicated by arrows 642, 643 and 644. Further, the arrows are shown as including only one or two arrows to indicate that this is a punch shot rather than a full swing.

Miscellaneous

Approach. The approach shot is a transition shot that is typically hit deep into the opponent's court to allow the player time to advance to the next. Any of the aforementioned shots can be utilized as long as the shot is properly aimed. To implement this type of a shot in the training module, the operator can state or place an indicator on the screen that this is an approach shot. As a result, the participant will attempt to make the indicated shot but aim for a deep location in the opponent's court.

Overhead Smash. The overhead smash is technically executed similar to a serve with the exception that the participant does not toss the ball, but instead waits for a lobbed shot or high bouncing shot to enter the hitting zone. As such, any of the serve screens (FIG. 6A, FIG. 6B or FIG. 6C) can be used for the overhead smash.

Moon Ball and Lob Shot. The moon ball and lob shot are technically executed in the same manner. They are shots intended to hit at the back of the opponent's court and are a high arching shot. For the lob shot, this is basically the same as a moon ball except it is intended to go over the opponent's head when they are at the net. The shots are typically struck similar to a forehand topspin or backhand topspin, although in some circumstances the flat or slice stroke may also be utilized. FIG. 6P is a conceptual design for presenting the forehand moon ball or lob shot skill training within a training module. In the illustrated screen 6P, the participant 602 sees the display of a target 625 about waist high. The target includes direction arrows 645 to assist the participant 602 in knowing what the swing path or type is. In this example, the arrows are pointing upwards and across the participant 602 to indicate the direction of the racquet movement. Additional arrows are included in the direction arrows 645 to indicate this shot is to go higher and farther back than normal shots.

Dropshot. The drop shot is basically a forehand or backhand slice or a forehand or backhand lobby slice. The only difference is that the dropshot is delivered when the opponent is at the back of their court. As such, to implement the dropshot in any of the training modules, the screens in FIG. 6F, FIG. 6I, FIG. 6L, or FIG. 6O can be utilized.

Sensory Input. As previously mentioned, the sensory input (256 in FIG. 2) can be implemented in a variety of manners. In some embodiments, the sensory input 256 is a camera type device that visually monitors and captures positions, directions, angles, etc. of the racquet. In addition, motion sensors may also be utilized. In such embodiments, targets may be utilized on one or more locations of the racquet and/or on one or more locations of the participant. The targets, which may be a bar code, a QR code, or simply a highly recognizable design that the camera or visual sensor can pick up and then track as the racquet or participant moves. By monitoring the position and size of the target, the sensory input 256 can determine the direction of movement, speed of movement, lateral movement, forward and backward movement, etc.

FIG. 7 is a still frame taken from a live racquet training system in operation. In the still frame, a racquet 704 is illustrated as including an attachment 706 on the face of the strings. The attachment 706 is a non-limiting example of a target that is used by a camera or visual system to monitor movement of the racquet. The presence of the blue target circle 708 indicates that the sensory input 256 has correctly detected and located the racquet 704.

In other embodiments, the racquet may include accelerometers, GPS receivers, RFIDs, gyroscopes etc. These devices can be attached to the racquet, and even locations on the body of the participant in some embodiments, and are used to identify motion, position, speed, direction, etc. of the racquet movements and then to transmit signals to the sensory input 256 indicative of this information.

Yet in other embodiments, a combination of any of these different techniques can be utilized. In addition, in some embodiments, multiple cameras or visual sensors can be placed the area that the participant stands in order to give more accurate motion information.

Visual Indicators for Swing Timing. As previously described, various targets may be presented on the screen to let the participant know where and how to swing. It should be appreciated that timing of a swing is a hugely important aspect in mastering the sport and skills of tennis. As a ball is approaching a tennis player, the player may let the ball bounce on the ground and then attempt to strike the ball as it is climbing back up. Alternatively, the tennis player may hit the ball in the air as it comes across the net (volley). There are many other situations that also exist, such as hitting an overhead shot for a lob or a moon ball, scooping a shot up off the ground before it is out of reach, etc. These and many other situations can be simulated with the racquet training system. As such, it is advantageous in some embodiments to use visual indicators, along with or as part of the targets to identify ball position and help the participant develop timing. For instance, in one embodiment, the targets may first appear at a low intensity level and rapidly increase in intensity to a maximum level before tapering back off to a low intensity level. In such an embodiment, the goal of the participant is to move the racquet to the target for a strike as close to the maximum intensity level as possible. It will be appreciated that this capability allows the operator to design multiple situations to test the participant by placing the targets at various positions, such as overhead, waist high, low, etc. and using the intensity to identify when the participant should hit the ball.

FIG. 8 illustrates diagram of screens for an exemplary implementation of an intensity-base target for showing the participant timing of the wing. At 800, the participant 602 is in the ready position understanding that the system is requiring the performance of a forehand topspin shot. The target 610 is illustrated as becoming more intense as the flow continues at 802 and 804. At 806 the target reaches its maximum intensity and then begins decreasing at 808, 810 and 812.

In some embodiments, a simulated ball may be used to show the flight path of the ball towards and through the target. This can be utilized along with or in lieu of varying the intensity of the targets. In general, when the flight or the path of the ball enters the target area, the participant should move the racquet into and through the target area. The target indicates where the participant's point of impact should be and the movement of the ball on the screen in real-time helps the participant to develop timing skills.

FIG. 9 is a timing diagram illustrating an exemplary implementation of using intensity on the target along with a moving ball to show the participant timing of the swing. As the ball is shown moving through the bounce path, when it enters the target area, the participant should swing the racquet through the target at the same time. At 900, the participant 602 is in the ready position understanding that the system is requiring the performance of a forehand topspin shot. The ball 980 is shown to be on a path to land in front of the participant 602. The target 610 is illustrated as becoming more intense as the flow continues at 902 and 904. During this time the ball 680 hits the court at 902 and then starts bouncing upwards toward the target 610. At 906 the target reaches its maximum intensity as the ball 608 enters the target area. At this point the participant 602 should swing and hit the ball. As such, the target 630 begins decreasing at 908, 910 and 912 but the ball 680 is no longer displayed as it was properly struck by the racquet.

Haptic Feedback. In some embodiments, haptic feedback can be used to provide real-time performance feedback to the participant. Those skilled in the art will understand that a tennis player can feel when he or she impacts a tennis ball on the sweet spot (middle of the racquet), on the outer edges of the string face, top edge of the string face, bottom edge of the string face, frame of the racquet, handle of the racquet, etc. The impact of the ball in these various areas feels different to the player. In some embodiments, actuators can be attached to the face of the racquet to provide haptic feedback that simulates the different ball strike feelings. The haptic feedback can be generated in a variety of ways that are known to those skilled in the art, including compressed air propulsion jets, vibrators, actuators, etc. The haptic feedback may also include audible feedback to simulate the various hits. With the ability of the sensory input 256 to accurately track the movement of the racquet and the timing and location of a simulated ball, the impact of the racquet and the ball, from a timing and a position perspective can be determined and the haptic feedback generated and delivered that is commensurate with the hit. The sensory input 256 is configured such that the granularity of the tracking of the racquet motion and position is in the millisecond range. Thus, the position, angle, direction, speed, etc. of the racquet can be updated every millisecond. This capability, along with the intrinsic knowledge of the simulated ball position or intensity of the target, enables the system to accurately measure the participant's performance on the shot.

Training Program Editor. The operator interface may also include a training program editor. The training program editor enables an operator 210 to select one or more training modules to be included in a training program. The training program editor may present a list of training modules available to the operator 210 and the operator can select the training modules to be included in a training program, select the order of the training modules, select the complexity of the training modules, etc. FIG. 10 is a conceptual illustration of a training program editor that could be employed in various embodiments of the training system. The operator interface screen 1000 includes a window showing the library of training modules 1002 that can be selected for a training program. In the illustrated embodiment, the library is shown as including Training Module A-n. Each training module may include, among other things, a name, such as Training Module A, and a description of the training module. Other information may also be included for the training module such as, the name of the creator, the length in time of the training module, a level of difficulty attributed to the training module, an indicator of the popularity of the training module by participants and/or operators, the number of times the training module has been included in a training program, etc.

The operator interface screen 1000 also includes a window to show the training program being created 1004. In the illustrated embodiment, the operator 210 has selected Training Module B, Training Module C, Training Module A and Training Module B again. As a non-limiting example, the operator can also set a MODE for a selected training module. The mode may identify the difficulty of the training module, such as EASY, INTERMEDIATE, PROFESSIONAL. As another example, the MODE for the training module may be listed at different levels such as YOUTH, HIGH SCHOOL, COLLEGE, PROFESSIONAL, etc. It should be appreciated that in some embodiments, the training modules may be created for certain skill levels and as such, the operator 210 can select training modules based on the identified skill level of the training module. The operator 210 may set the number of repetitions (REPS) for the training module, such as the number of times that the training module will repeat or the amount of time that a training module will operate before going to the next training module. In some embodiments, the REPS may also include a category of OPERATOR CONTROL, indicating that the training module will continue to be active until the operator takes an action to move to the next training module, or PERFORMANCE CONTROL, indicating that the training module will continue until certain performance criteria are met. The performance criteria may include, as a non-limiting example, a requirement that a certain percentage of the participants must reach a threshold accuracy level before the training module stops. As a non-limiting example, 20% of the participants need to be performing at 90% accuracy level in the physical activity before a transition can occur. Other embodiments may include other features such as entering notes that can be displayed only to the operator 210 to help guide the training session, entering notes and words of encouragement to be displayed or audibly played to the participants, selecting background music to be played for the training program or the training modules, etc.

In the non-limiting operator interface example, the operator is presented with a variety of controls that the operator can use in creating the training program. Initially, the operator may begin the creation of a training program by selecting the NEW control 1020. The operator can then add training modules to the training program by selecting a training module from the training module library 1002 and dragging it to the training program window 1004 or actuating the ADD control 1012. Likewise, the operator can select a training module that is listed in the training program window 1004 and either drag it to the trash can 1022 or actuate the remove (RMV) control 1014.

The operator can change the order of the training modules in the training program by selecting a training module in the training program window 404 and dragging it up or down in the list. Alternatively, the operator can select a training module and actuate the UP control 1016 or DOWN control 1018 to move the training module up or down in the list.

When the operator is satisfied with the training program, the operator can select the SAVE control 1010 to save the training program for later use. The operator 210 may then be requested to provide a name for the training program and may also be allowed to set access and modification privileges for the training program. It should be appreciated that the operator 210 can also select the OPEN control 1024 to pull up a list of already created training programs that can be edited or deleted.

Training Program Previewer. The operator interface may also include a training program previewer. The training program previewer can allow the operator to run through a training program, such as to simulate the presentment of the training program, to determine the length of time that the training program will run, observe the flow of the training program determine the difficulty of the training program, take notes on instructions and script the training program, etc. Embodiments may allow the operator to pause, fast forward, exit the training program or switch to training program editing mode from the previewer. It should be appreciated that the training program previewer may also work on the training module level, thus allowing an operator 210 to preview a training module before adding it into a training program.

Program Executor. The program executor of the server application 216 enables the operator 210 to select a training program and then launch and present the training program to the participants. FIG. 11 is a flow diagram illustrating the operation of the system after launching a training program. The flow of operation 1100 will also be described with reference to the system components presented in FIG. 2. Initially, the operator 210 selects a training program 1110 from the list of available training programs available on the server 208. An initial screen can then be presented to the operator 210 on the display device 214 to show that the training program is ready to begin. When launching, the operator 210 may have the option to select the time that the training program will begin. The server application 216, upon the launching of the training program, may also provide a push indication or notice to one or more user systems 250 that the training program is about to begin. Alternatively, or in addition to, the user systems 250 can connect to the in-studio system 200 devices to identify a schedule for upcoming training programs and then select a training program in which to participate. It should be appreciated that the server application 216 can provide the push notifications to the user systems 250 by either directly communicating with the user systems 250 or be sending messages such as emails, text messages, SMS messages, etc. to notify the participants that a training program is about to begin or to invite participants to a training program. In any of these scenarios, the server application 216 may include a period of time for various participants to join in to the training program 1112. Once a user system 250 has joined into a training program, a screen can be presented to the user device 252 to indicate who all is joined into the training program and when the training program will commence.

As participants are joining into the training program, the server application 216 may present this information 1114 to the operator 210 on the display device 214. The information presented may include the name or username of the participant, a picture of the participant and other information about the participant such as skill level, number of times they have participated in the training program etc. In addition, the server application 216 may also transmit this information, or a subset of this information, to the user systems 250 so that other participants can see who they are participating with.

Once the participants have joined into the training program, the training program can commence 1116. The commencement of the training program may be triggered by a certain time of day, an action by the operator 210, a limit of the number of participants being reached, etc. as well as a combination of two or more such triggers.

When the training program commences, a first training module from the list of training modules in the training program is selected by the server application 216. If the training program only includes one training module, that training module is run and then the training program is complete. However, for training programs that include multiple training modules, the server application 216 enters a loop 1118 in which the training modules are performed serially until the entire training program has been completed or otherwise stopped or paused by the operator.

In the illustrated flow, it is assumed that the training program includes “n” training modules. The server application 216 then selects the first training module 1120 and then enters the demonstration mode for the training module 1122. During the demonstration mode, the simulated physical activity of the training module can be presented on the display device 214 of the server 208 and the operator 210 can perform the physical activity. The camera 204 and the recorder 202 can collect video actions and voice instructions of the operator 210 and then record the audio/video content and store it in the server 208 for non-real-time systems and/or stream the audio/video for real-time systems to the user systems 250 over the network 280.

Each of the participants that have joined the training program can receive the streaming audio/video in real-time and the video can be presented on the screen 254 and speaker 259 by the user application 260. The physical requirements, such as target positions etc. as described in conjunction with FIG. 4 can be overlaid on the streaming video from the operator and/or displayed on the display device 214 for the operator 210.

Once the demonstration is complete, the server application 216 transitions to the trial mode 1124. The transition to the trial mode can be triggered in a variety of manners. In one embodiment, the operator 210 may actuate the actuator 206 to trigger the transition. In other embodiments, the transition may be triggered after a certain amount of time, a certain number of repetitions of the physical activity, a voice command from the operator 210, etc. In transitioning to the trial mode, the server application 216 sends a control signal to each of the user system 250 to indicate that the trial mode is being entered.

Upon reception of the control signal from the server application 216, the user application 260 in each of the user devices 252 also enable the trial mode. In enabling the trial mode at the user systems 250, the user application 260 receives the training module video data or other data from the server application 216 and begins rendering it upon the screens 254 of each user device 250 in a synchronized manner. Thus, all participants are synchronized in performing the physical activity. The user application 260 also enables the camera 256 to begin tracking the physical activity of the user 262 and/or the object 264 utilized by the user in view of the definition of the physical activity in the training module. For example, the training module may have certain target positions and motion requirements that the user must satisfy, such as moving a ball from one location to another location at a certain speed. The user application 260 is able to synchronize the training module with the activity of the user 262 and object 264 to identify if the user 262 is satisfying the physical criteria. A detailed description of this aspect of the user application 260 and its configuration for identifying a moving object in a video and mapping the motion of the object to a particular drill in a training module can be seen in U.S. Pat. No. 9,275,470B1 which is above-incorporated by reference. The present solution expands the functionality of the invention presented in U.S. Pat. No. 9,275,470B1 such that live classes may be conducted with a plurality of remote users over the Internet. The operator “controls” the user application 216 of each remotely connected user device 252 via signals sent over the live stream connection. In this way, the operator 210 may guide a plurality of remote participants through a series of drills in a workout. Further, the server application 216 may also send a timing control signal to ensure that each of the user systems 250 remain in sync.

During the trial mode, the user application 260 gathers performance data pertaining to the physical activities of the user 262. In the various embodiments, this performance data can be processed by the user application 260 and/or the server application 216 to rank and score the performance of the user. Many techniques can be utilized in ranking the user performance. As a non-limiting example, looking at the forehand and backhand training module as presented in FIG. 3, the user performance can be ranked based on the number of times that the user 262 was able to position the racquet within the required target area within the allotted time frame or when the indicator shows that the ball is ready to be hit. For example, in Screen A, the training module may allot a time t1 for the user 262 to transition the racquet 264 from ready position to the hitting position, time t2 to transition the racquet 264 back to the ready position, in Screen B, time t3 to transition the racquet 264 from the ready position to the hitting position, and time t4 to transition the racquet 264 from back to the ready position. The ranking can take into consideration the accuracy of the user 262 in hitting the target positions 312 and 316 and moving in the required directions 314 and 318 respectively illustrated in FIG. 3 for the training module as well as the accuracy of meeting the time allocations. As a non-limiting example, the participant 262 may be credited with a single point by getting the object 264 into the target position 312 within the allotted time. If the participant 262 is slightly off the target position 312, the point may be withheld or reduced depending on the severity of missing the targeted requirements. If the user 262 hits the location target 312 but misses the time target, the user 262 may be credited with only a half point. If the user meets the time target but is slightly off on the positioning target, the user may be credited with ¾ of a point. If the user meets the time target and totally misses the positioning target, the user may be credited with ¼ of a point. During the trial mode, the user application 260 can tally all the points for the user and then provide a point total to the server application 216 at the end of the trial mode. In other embodiments, the user application 260 may only gather information about the performance of the user 262 and transmit this data to the server application 216 for ranking and scoring the performance of the participant 262. It should be appreciated that other ranking techniques may be applied and more or fewer performance criteria parameters may be utilized in such ranking techniques and the present application is not limited to any particular ranking technique.

Once the trial mode is completed, the server application 216 causes a transition to the results mode 1126. The transition to the results mode can be triggered in a variety of manners. In one embodiment, the operator 210 may actuate the actuator 206 to trigger the transition. In other embodiments, the transition may be triggered after a certain amount of time, a certain number of repetitions of the physical activity, as well as other triggering events described herein. In transitioning to the results mode, the server application 216 sends a control signal to each of the user system 250 to indicate that the results mode is being entered.

In the results mode, the server application 216 can process the performance data received from each of the user systems 250 that participated in the training module. The server system 216 can rank the participants based on their score and then present the information on the display device 214 of the server 208 and/or transmit the information to each of the user systems 250, where the user application 260 can display the results on the screen 254 of the user device 252.

Once the results have been presented, the server application 216 can then examine the training program again to determine if there is another training module to be processed 1118. If another training module is in the training program, the server application can select and load the next training module 1120 and continue processing the training module in the loop. Otherwise, if all of the training modules have been processed, then the training program can be terminated.

It should be appreciated that while the demonstration mode, trial mode and results mode have all been described as a state diagram in which only one state exists or is active at a time. However, in some embodiments, the modes may overlap or be coexistent. For instance, in one embodiment, the operator 210 can perform the physical activities of the demonstration mode and the server application 216 can stream the audio/video of the operator 210 to the user systems 250. Simultaneously, the user systems 250 may present the audio/video stream of the operator 210 on a portion of the screen 254 and present the trial mode screens or video on another portion of the screen 254. In such embodiments, the user 262 can watch the actions of the operator 210 while the user 262 is performing the actions as well. In other embodiments, the user application 260 may gather performance data during the trial mode and immediately send the performance data to the server application 216. The server application 216 can then start ranking the performance of the users 262 and displaying performance results on the display device 214 and transmitting the performance results to the user systems 250 where the user application 260 can present the rankings on the screen 254. Thus, the trial mode and the results mode may also be coexistent. In yet another embodiment, the demonstration mode, trial mode and results mode may all be coexistent.

Synchronization and Command Control.

Another aspect of the various embodiments of the present invention is the use of embedded data within the video signal or video stream that is sent to the various user devices to provide synchronization of the user devices with the video stream and to otherwise control the user devices. Those skilled in the relevant art will be aware that the video streamed to the user devices may traverse different paths. Some of these paths may be over wired networks, wireless networks or a combination of both. Further, some of these paths may be more congested, lossy, or have less bandwidth etc. than the other paths. As such, the buffering applied to the various communication paths leading to the various user devices will inevitably result in the user systems receiving and rendering the video out of synchronization with each other. Because of this fact, the use of a timer is not a practical solution for ensuring that the user devices are synchronized with the video feed. Thus, the various embodiments may embed data or commands within the video feed to provide synchronization of each user device with the video feed.

In an exemplary embodiment, each training module includes or defines multiple phases or modes. As previously described, an embodiment may utilize a training module that first includes a demonstration phase, followed by a trial phase, then followed by a results phase and finally a leaderboard phase. During each of these phases, the user devices need to operate in different modes (i.e., demonstration mode, trial mode, results mode, leaderboard mode respectively). For example, when a training module begins, the user devices need to be set to demonstration mode in which the user devices receive a live video stream and then render it on a display device. When the training module enters the trial phase, the user devices need to change what is being displayed (i.e. present the live instructor feed in one window and the positioning targets video in another window or stop presenting the live feed and switch to the positioning targets video) and the user devices need to activate and control the camera to monitor the participant activities. Further, when the training module enters the results phase, the user devices need to switch to the results mode in which the user devices send the collected motion data to the server and then receive results data. Finally, when the training module enters the leaderboard phase, the user devices need to transition to the leaderboard mode to display the newly received content.

In some embodiments, the synchronization of the modes of the user devices to the training module phases is accomplished by embedding data signals within the video data that is transmitted to the user devices. FIG. 12 is a signal diagram representative of the use of embedded data commands to provide synchronization of the user devices with the video content and server application for various embodiments. In the illustrated non-limiting example, training module A 1200a is being processed. The enter demonstration mode command 1202a is embedded at the beginning of the demonstration phase 1204a by embedding it within the video stream to the user devices. The demonstration mode command 1202a operates to place the user systems 250 and user device 252 into demonstration mode. When the training module A 1200a enters the trial phase, the enter trial mode command 1206a is embedded into the streaming video at the beginning of the performance targets phase 1208a (or the end of the demonstration phase 1206a) to cause the user systems 250 to switch to trial mode. Likewise, when the training module A 1200a enters the results phase, the enter results mode command 1210a is sent at the beginning of the results video phase 1212a (or the end of the performance targets video content 1208a) to cause the user systems 250 to switch to the results mode. Finally, when the training module A 1200a enters the leaderboard phase, the enter leaderboard mode command 1214a is sent at the beginning of the leaderboard video phase 1216a (or the end of the results video content 1212a) to cause the user systems 250 to enter the leaderboard mode. At this point, if additional training modules are available in the training program, the enter demonstration mode command 1202b is sent at the beginning of the demonstration video content 1204b (or the end of the leaderboard phase 1216b) to cause the user systems 250 to transition back to the demonstration mode. Thus, it will be appreciated that as the user systems 250 receive the video content, they are synchronized to the video content by the embedded commands that trigger them to switch to the correct mode of operation. Using broken lines, it is illustrated that the data commands can be embedded at the beginning of a phase, as well as any point during the phase. As a non-limiting example, the demonstration mode command 1202a can be sent at the beginning of the demonstration phase and/or at various points of time during the demonstration phase. Preferably, the demonstration mode command 1202a is included at the beginning of the demonstration phase 1204a, and then at intervals during the demonstration phase 1204a.

In some embodiments, the operator and/or the in-studio system can send other commands to user systems 250 to further control the operation of the user systems. As a non-limiting example, the operator 210 can control what the user application 260 is presenting on the screen 254 of the user device 252. Thus, the operator can cause the user applications 260 to only display the training module operation and the video of the user performing the required physical activities. If the operator 210 sees that participants are having trouble, the operator 210 can send a command to one or more of the user systems 250 to cause the user application 260 to receive and display the live stream of the operator 210 performing the physical activity or to replay the original video obtained during the demonstration mode. This can be displayed by itself or juxtaposed on the screen 254 with the training module and video of the user.

In some embodiments, the operator 210 may also be able to send a command to a user system 250 to direct the user application 260 to not only monitor the participant, but to also stream the video, and maybe audio, of the participant to the server system 200. Thus, if the user application 260 detects that a participant is not scoring at least at a minimum threshold level, the user application 260 may send a notification to the server application 216 to indicate that the participant may need help. The server system may flag this information on the display device 214 so that the operator 210 is made aware. The operator 210 may then choose to have the server application 216 send a command to the user system 250 forcing the user system 250 to stream the audio/video of the participant. The server application 216 can then render this live stream on all or a portion of the display device 214 for the operator 210 to observe. The operator 210 can then give audible and/or visual feedback to the participant 262 through the server system 200.

Similarly, in some embodiments, the server application 216 may not only be able to receive the real-time stream of a participant but also may be able to rebroadcast the stream to other user devices 250. Thus, if a certain participant is excelling on a physical activity, the operator 210 can have that participant's live feed streamed to other participants as an example. Or, if the participant is experiencing a certain problem, the operator 210 can stream the participant's video as an example to other participants about techniques to avoid.

The present invention has been described using detailed descriptions of embodiments thereof that are provided by way of example and are not intended to limit the scope of the invention. The described embodiments comprise different features, not all of which are required in all embodiments of the invention. Some embodiments of the present invention utilize only some of the features or possible combinations of the features. Variations of embodiments of the present invention that are described and embodiments of the present invention comprising different combinations of features noted in the described embodiments will occur to persons of the art.

It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described herein above. Rather the scope of the invention is defined by the claims that follow.