SYSTEM FOR TRACKING SHOOTING PERFORMANCE INCLUDING A LIVE CAMERA FOR CALCULATING SCORING

Description

BACKGROUND

Target shooting has always been a popular way to use firearms in a safe and controlled manner that allows for a person to analyze and hone their shooting skills. Target shooting can be done on a gun range or out in an open area. The problem with analyzing target shooting is that the feedback is either delayed, hard to judge as the accuracy of the shots and the shooter's actions, or both. In target shooting, there is usually an object that the shooter is aiming for, such as a paper, wood or metal target. The shooter has to retrieve the target after they are done with their target practice. This doesn't allow for instant feedback that would allow for a shooter to adjust their aim between shots based on the feedback as they won't know how well they shot until after they are done shooting multiple rounds.

DESCRIPTION OF RELATED ART

One way to overcome this lack of real-time feedback is for shooting at a range with a permanent camera installed that can track where the shooter is shooting. The problem with these systems is that they are expensive and complex to install which limits their accessibility to most shooters. These permanent camera systems cannot provide real-time feedback to a shooter's personal device, which would allow for personalization of the scoring metrics chosen by the shooter. Another disadvantage of these systems is that they require predetermined targets so the system can track whether a bullet has struck the target.

RELEVANT PRIOR ART

US Publication 2022/0326596 discloses such a camera based shooting accuracy feedback system. The system uses a camera with multiple sensors for capturing target image data. The targets must be pre-programmed into the system or they need to be tagged with a sticker or QR Code, so that system can recognize the targets. The multiple sensors and need to predefine or tag the targets makes this system more expensive and cumbersome to set up.

US Publication 2012/0258432 (now abandoned) discloses a camera based shooting accuracy feedback system that uses a camera to detect when and where a projective enters a target. The targets require a calibration mark to be placed on the targets to allow for the camera to know the size and configuration of the target, unlike the target in the invention. The system found in the application also does not use a recoil detector, unlike the shot detection performed in the application.

SUMMARY OF THE INVENTION

The overall scope of the invention is to provide a person with a system for accurately tracking and analyzing their shooting in real time. The system would include a recoil detection device that is small enough to be attached to a shooter's finger, but could also be placed on a wrist or arm, or even the firearm itself. The recoil detection device would communicate with a camera system for recording an analyzing the target and the shots made or missed by the shooter.

The recoil detection device contains an accelerometer, a gyroscope, and a magnetometer to detect when the shooter pulls the trigger and firearm discharges a bullet. While the device is small, it also needs to be rugged enough to withstand the recoil of the firearm, all types of weather conditions, and have enough battery life to last for an entire shooting session.

The camera system would consist of a portable camera sensor and a monocular for aiming the camera at the target or targets. The monocular could use a fixed focal length lens or a telephoto or zoom lens for aiming the camera sensor at distant targets. The camera system could use an automatic target selecting system for determining each target in a frame and applying the scoring algorithm to the portions of the video and only those portions. This would allow for a reduction of processing power as the camera system would not need to track objects in the entire captured video. The camera sensor could be placed on a portable tripod, which could contain orientation motors for aiming and focusing the camera system on a target(s).

The camera system contains wireless connectivity to connect it to a smartphone, tablet, computer, or any type of wireless computing device. This connectivity is used by the device to send and receive information to and from the computing device. The connectivity could also be used to communicate with other sensors, such as a recoil detector and/or a timer device. When these devices are used in unison, they aggregate the information from the various sensors and provide the shooter with instant feedback as to their shooting performance including but not limited to target feedback, video/image feedback, timing information, wind speed, humidity, and temperature.

The scoring system would allow for shooters to view their scores in real-time alongside other shooters anywhere on earth, which would allow for shooters to take part in global shooting competitions. The competitions could be of any scale or duration, such as local, county, state, region, or even household. The shooter could create their own competitions that have customizable rules allowing for endless types of contests. The competitions could also be team based, where the teammates need not be in the same place to compete.

To protect the shooter's information the communication is encrypted between the sensor devices and the computing device. This prevents nearby shooters using the same system for interfering with the shooter's feedback. While it is preferable for all of the sensor devices to be used in unison, any number or combination of sensor devices could be used along with the recoil detector to give the shooter any feedback they require.

Multiple camera systems could be used in unison to aiming at multiple targets on different areas on the range. This would allow for the shooter to move around on a range or shooting field to add another level of difficulty to test the shooter's ability. Besides being used for target practice for recreational shooters, it could also be used for law enforcement and military training. The person conducting the training could use the real-time feedback to see how the trainee is doing and instruct the shooter on how to improve in the middle of an exercise.

While the preferred embodiment of the shooting tracking system would use the recoil detecting device, a sound based shot detector could also be used. This detector would be cheaper than the recoil detection device, which would allow for more shooters to be able to participate. While the invention will be described using regular firearms, air guns shooting plastic or metal projectiles, and bows could be tracked using the system.

BRIEF DESCRIPTIONS OF THE FIGURES

FIG. 1 shows the recoil detector attached to a user's finger.

FIG. 2 shows an overall embodiment of the targeting system.

FIG. 3 shows the recoil detector mounted to a finger or firearm.

FIG. 4 shows a phone for running the app.

FIG. 5 shows an example of a GUI running on the app.

FIG. 6 shows an embodiment of the camera system.

FIG. 7 shows a detailed view of the timing information.

DETAILED DESCRIPTION

FIG. 1 shows a preferred embodiment of the invention showing a person holding a pistol with their right hand, although the invention could be used on either hand. The recoil detecting device 102 would be attached to the user's finger via a strap 103, while they hold onto pistol 101. While a pistol will be shown in the figures, any type of firearm/shooting device could be used with the recoil detection device, including but not limited to a shotgun, a rifle, a bow, a revolver, a grenade launcher, air-gun, and a blow dart. The recoil device is shown as being attached to the user's finger, but the recoil device could be, but is not limited to being, attached to a user's wrist, arm, body, or the firearm itself.

The details of the recoil detection device can be found in application Ser. No. 18/454,871 which is incorporated by reference.

FIG. 2 shows an overview of the shot tracking system. The recoil detection device 102 is attached to the firearm 101 for detecting when the user fires the firearm 101. Information from the recoil device 102 is sent and received from the user's phone 206. The recoil information generated by the recoil device is sent to the phone to initiate a timer system to start a timer, which would indicate how long it takes a user to aim and shoot the firearm, along with how much time elapses between the firings. The phone also receives and sends information to the camera system 201. The recoil information can also be sent to the camera system directly from the recoil device or via the phone. The recoil information would trigger the camera system, which is aimed at the target, to start recording video or taking a series of still images that are relayed back to the phone. The phone uses an algorithm to analyze the video or still images to create an accuracy score that is displayed to the user on the phone. The information displayed to the user consists of, but is not limited to, image or where the bullet struck the target, an accuracy score based on a general scoring system or one that they have customized, and the timing information generated by the timer system. The information from the camera system would be used to determine when the bullet struck the target to determine the bullet travel time.

The shot tracking system uses all the information sent and received from/by the phone, recoil detection device, timer system, and the camera system to generate feedback for the user. This information could be aggregated by using AI or machine learning to learn the specifics of the user's shooting information and generate customized information to display to the user on the phone. The processing of the information could be performed locally on the phone or the phone could upload the information to a server to perform the processing, the information being sent from the server to the phone. The real-time feedback could be a numerical score using default scoring rules, customized rules inputted by the user, automatically generated by the phone, or any combination. The feedback could be sent to other user's phones that are using the same system to create a contest or leaderboard for all that are participating.

The camera system 201 would include a scope 203, with a lens or lenses, and a sensor 202. The sensor includes an eye piece that allows the user to aim the camera system at the target, but the camera system could also include an auto aiming feature where a motorized mount is used to hold the camera system, which would use image recognition to detect, focus, and aim at the target. The camera system uses edge detection, contour tracing, and shape matching for detecting when a bullet strikes the target. The phone could display new bullet strikes with a particular shape or color to differentiate them from the previous bullet strikes on the target. The camera system would also be able to determine when there is not a bullet strike, which would be outputted to the phone in the form of a miss alert and/or how far off target the bullet was from the target.

The shot timer system provides information indicating how long it takes the user to shoot. This information includes, but is not limited to, time to aim and shoot the firearm, time for the bullet to reach the target after the firearm was shot, the time it takes for the user to draw the weapon from a holster or other position, time between two or more shots, or any combination of these times. The feedback from the timer system would help a user work on their shooting speed, which combined with accuracy, would present to the user the most complete coaching information on the phone. The recoil system could use AI or Machine Learning to determine a pattern for the user's shooting style to enable the system to start the timer automatically when the user move's the firearm.

The shot tracking system provides the user with the ability to share the information on social network accounts, overlay data over the images, and connect the shooter to e-commerce sites to buy different firearms or accessories. The system could include a shooting coach feature that aggregates all the information from the various systems to provide the user with real-time adjustments to make to their stance, how the user holds the firearm, aim, and any other information that would help the user improve their shooting. Multiple camera systems could be used to capture multiple images of the target, including but not limited to, different angles in all three dimensions, an angle that shows the user, an angle that shows the user and the back of the target, angles that show the target from the sides, top or bottom. The shot tracking system could also use voice recognition software to allow for the user to control the system without taking their hand off of the firearm. The camera system could use any known and future type of image processing to enhance the images captured. The system could process the information locally or it could send and receive information from a cloud based system to reduce the processing requirements of the phone.

FIG. 3 shows the recoil detecting device that is attached to the user's finger or the firearm. The strap 103 is preferably made of a sturdy plastic or rubber, but could also be made out of any type of fabric. The strap is easily connected and disconnect from the recoil device to allow for the device to be customized with different accessories in any type of color or pattern and to allow for the strap and device to be cleaned.

FIG. 4 shows an example of a user device, a phone that could run the shot tracking system app, but the user device could be a tablet, computer, laptop, watch, VR headset, AR headset, touchscreen TV, or any other device that can display data and be interacted with by a user. FIG. 5 shows an example of a GUI running on the app. The GUI displays a leader board update, which would show a live ranking of different shooter's scores. The shooters could be ranked by their overall score from multiple days or could be ranked for their scores for the current day. There could also be rankings that based on a specific contest for a single or multiple rounds.

FIG. 6 shows a disassembled view of the camera system. The camera sensor 601 that is preferably a CCD sensor, but any type of sensor with the ability to convert light information into digital signals could be used. Coupler 602 would connect the camera sensor to the scope 603. While shown as separate components, the camera system could be a single component or any number of separate components. The separate component design would be preferred as it would allow for each component to be upgraded separately, like a higher pixel count sensor and/or a different kind of lens. The lens could be a fixed focal length which would allow for a smaller scope to be used, or the scope could contain a zoom or telephoto lens that would allow for different length ranges to be shot on.

FIG. 7 shows examples of different detailed information displayed to the user via the phone. Screen 701 shows the detailed information for a single shooting session including, but not limited to, if the session is indoors or outdoors, time of the session, date of the session, the number of the session, the time to take a single shot or multiple shots, total shooting time, the split time, how many rounds were fired in a customizable window of time, and buttons to pause, resume, and stop the session. Screen 702 shows an interface for comparing different sessions that includes the options to delete sessions, display details for a single session, and the par timer for a session.

All the components of the system would preferably be compact, portable, and battery powered to allow for easy transport and setup of the system. All the components are designed to work best when used in unison, but each of the components could be used independently of each other. This is possible as all the components would preferably use wireless protocols, mentioned above, so that once powered up, all the devices would automatically find and pair with each other.

The shooter can either set up the camera system each time that it is used, or it can be permanently installed so that it can be used at any time with minimal setup. Using the as needed example, the shooter would set up the camera system on a tripod, the tripod could be any commercially available camera tripod or any device that is able to hold a camera steady and focused on a target placed by the shooter. The tripod could be fixed, as it would be aimed by the shooter once and then the camera system would focus on the target and automatically detected the area in the frame that would be monitored. In another embodiment the tripod could contain motors that could aim the camera at a target using both horizontal and vertical adjustments, so as the shooter would just need to aim the camera at the general area of the target, and the camera system would instruct the motors to precisely aim at the target without any further interaction from the shooter. The aiming and focusing could be manually adjusted by the shooter using an eyepiece or scope on the camera system.

The aiming of the camera system could be performed by the camera system entirely by the camera system without the need for any other devices, or the camera system could communicate with the user device to control all of the movements and focusing of the camera system.

After the camera system is aimed at the target, the camera system will do image analysis on the target to create a baseline mask for the target, by evaluating if there are any previous holes in the target from a previous shooting session so the camera system knows to ignore the previous holes and only analyze the holes in the target made during the current shooting session. Once this mask image is created the camera system waits for the recoil trigger system, or any system that detects when a gun is fired, to trigger the analysis of the target. The camera system could use a continuous recording of the target so the shooter can go back and review the entire video (or periodic still frames) to review the shooting session. The camera system could also use moving window or circular buffer so that when a shot is detected, the camera system records the moments before a shot is fired and immediately afterwards. The window could be user a default length of time, a customized length of time, or limited automatically by the storage available on the camera system or the user device. The video/image captured by the camera system could be live streamed to the user device, a remote device, or anywhere other device that is connected to the internet.

When a shot is detected, the camera system analyzes video/image data captured and uses a mask to compare the current target to the one stored by the camera system, or the current target can be compared to the captured video/image from the previous shot. When a shot is fired and the camera system does not detect a change, the system indicates that the shot missed the target, which is shared with the other local network connected components for creating a score for a shooter. If a change in the video/image of the target is detected after the shot is detected, then the camera system analyzes the stored video/image to determine where on the target the projectile has struck. Once the x and y coordinates are determined for the current target strike, the camera system compares the coordinates of the strike to shooter defined, remote shooter defined, or remotely defined by the system to the scoring areas of the target. This would allow for the system to assign a score to each time the target is hit or missed by the projectile. These scores could be numerical and presented on a simple list, an overlay on a virtual target, or the image of the actual target. While it is preferred to use a new target, if the target is paper or cardboard, for each shooting session, the shooter could use the same target for multiple sessions as the image analysis would detect that a projectile has hit an area before and detect when a projectile goes through a previous hole in the target. This enables the camera system to detect the size placed in the target, which would enable the system to identify what caliber bullets were used.

The image processing performed by the camera system has been described as a mask based system, any type of image processing could be used to analyze the captured video/images. Types of processing that could be used include, but don't exclude, Fourier Transforms, FFTs, RAW image processing, compressed image processing, and enhancing the captured images before the processing is performed, or any combination of these techniques. When using the mask process, the image may be turned into a binary image consisting of a black background and white holes in the target so the system could score the shooting more accurately and faster. AI and Machine Learning could be used to teach the camera system to identify the holes in the targets faster and more efficiently.

The shot tracking and scoring system could be used for personal shooting training, law enforcement or military training, teaching an inexperienced user how to shoot safely, or any other firearm related training. Information presented to the user on the phone could include, but is not limited to, ammunition tracking, video analysis, scenario based training, health warnings such as a warning about the user's hearing, maintenance reminders for the firearm, and in app purchases for adding software and/or hardware capabilities to the shot tracking system.

The devices mentioned above could be implemented using any type of processor architecture able to execute software including, but not limited to, x86, ENIAC, RISC, Pentium™, and Apple Silicon™. The software could be any type of code that is used to instruct a processor to perform instructions including, but not limited to, Python™, Java™, C+™, FORTRAN, and Assembly. The software could be stored on any type of non-transitory medium including, but not limited to, RAM, ROM, Flash Memory, Punch Cards, Piano Player Reels, Hard Drives, and physical servers.