Radar-based object detection for vehicles

Description

PRIORITY

This application claims priority to U.S. Provisional Patent Application Ser. No. 62/237,975 filed on Oct. 6, 2015, the disclosure of which is incorporated by reference herein in its entirety.

BACKGROUND

This background description is provided for the purpose of generally presenting the context of the disclosure. Unless otherwise indicated herein, material described in this section is neither expressly nor impliedly admitted to be prior art to the present disclosure or the appended claims.

Vehicles, such as automobiles, boats, or aircrafts, can be dangerous when the driver of the vehicle fails to pay attention to driving. This may also be the case in autonomous driving experiences, where the driver may be relied upon as a fallback mechanism in the event that the autonomous driving system fails or is unable to handle a particular type of navigation.

SUMMARY

This document describes techniques and devices for radar-based object detection for vehicles. The techniques describe a radar-based object detection component implemented in a vehicle that is configured to detect characteristics of persons within the vehicle, such as a driver or other passengers. Then, based on the detected characteristics, an activity of the person can be determined and various operations can be initiated based on the activity, such as initiating a warning when the driver is not paying attention to driving, automatically slowing down the vehicle, and so forth. In some cases, the radar-based object detection component can also be implemented to detect characteristics of objects positioned external to the vehicle, such as pedestrians, other vehicles, foreign objects in the road, and so forth. The radar-based object detection component may also be implemented to authenticate a driver of the vehicle, such as by detecting biometric characteristics of the driver or recognizing a series of gestures corresponding to an authentication sequence. This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Various aspects of radar-based object detection for vehicles are described with reference to the following drawings. The same numbers are used throughout the drawings to reference like features and components:

FIG. 1 is an illustration of an environment in an example implementation that is operable to employ radar-based object detection for a vehicle.

FIG. 2 illustrates an example implementation of the vehicle computing system of FIG. 1 in greater detail.

FIG. 3 illustrates an example of RF wave propagation, and a corresponding reflected wave propagation.

FIG. 4 illustrates an example environment in which multiple antenna are used to ascertain information about a target object.

FIG. 5 is a flow diagram depicting a procedure in an example implementation.

FIG. 6 illustrates various components of an example vehicle computing system that incorporates radar-based object detection for vehicles as described with reference to FIGS. 1-5.

DETAILED DESCRIPTION

Overview

This document describes techniques and devices for radar-based object detection for vehicles. The techniques describe a radar-based object detection component for a vehicle (e.g., an automobile, boat, or plane) that is configured to detect various characteristics of persons with the vehicle (e.g., a driver and passengers), as well as characteristics of objects external to the vehicle (e.g., pedestrians, other vehicles, or foreign objects in the road).

For example, the radar-based object detection component can monitor a presence and attention level of a driver of the vehicle while the vehicle is moving by initiating transmission of an outgoing RF signal via a radar-emitting element of a radar sensor, receiving, via an antenna of the radar sensor, an incoming RF signal generated by the outgoing RF signal reflecting off the driver of the vehicle, and analyzing the incoming RF signal to detect one or more characteristics of the driver. Such characteristics, for example, can include a position and movement of the driver's body or a specific body part, such as the driver's hands, mouth, eyes, and so forth.

Then, based on the detected characteristics, the radar-based object detection component can determine an activity of the driver. As described herein, an activity of the driver corresponds to an activity currently being performed by the driver, such as driving with one or both hands on the steering wheel, being awake, drowsy, or asleep, interacting with a mobile device (e.g., texting), looking straight ahead, sideways, or backwards, talking (e.g., to a passenger in the vehicle or during a phone call), sitting somewhere other than the driver's seat (e.g., in an autonomous driving experience), looking in the glove compartment, and so forth. Based on the determined activity, one or more operations can be initiated. Generally, the operations improve the driving experience, increase the safety of the driving experience, provide security for the vehicle, or control navigation of the vehicle.

For example, in some cases, the radar-based object detection component determines an attention level of the driver, based on the activity, and then initiates the one or more operations based on the determined attention level. For example, activities such as texting, looking backwards, or being drowsy or sleepy, may be indicative of a low attention level. In contrast, activities such as driving with both hands on the wheel are indicative of a high attention level. Thus, in some cases, a warning (e.g., audible, visual, or tactile) may be initiated, in response to determining that the driver has a low attention level, in order to alert the driver to pay attention to driving. In this way, the radar-based object detection component monitors the driver without requiring the driver's deliberate or conscious interaction with the system.

In one or more implementations, the radar-based object detection component monitors the presence and attention level of the driver in an autonomous or semi-autonomous vehicle system. Autonomous vehicles are developed to navigate and operate either unmanned or to assist a vehicle operator, and can utilize many different types of sensors, automation, robotics, and other computer-controlled systems and mechanisms. However, in many cases, the driver must still act as a “fallback mechanism” in order to handle driving duties in certain instances where the autonomous system fails or is unable to control navigation. In these cases, the radar-based object detection component may monitor the driver's presence and attention level to ensure that the driver is a suitable backup in the event that the autonomous system needs to switch over to the manual system. In the event that the driver's attention is low, or the driver is not present in the driver's seat, the component may initiate various warnings to ensure that the driver is reminded that he may be needed as a fallback mechanism. Furthermore, in some cases, the radar-based object detection component may prevent a transition from an autonomous driving mode to a manual driving mode if the attention level of the driver is below a threshold indicating that the driver is not paying attention or is not present in the driver's seat. As another example, if the system is in autonomous driving mode or cruise control mode, the component may detect if the driver has a low attention level, and in response, cause the vehicle to slow down while at the same time alerting the driver.

Similar techniques may also be applied to passengers within the vehicle. For example, radar-emitting elements and antennas may be positioned in the rear of the vehicle to monitor a toddler or baby, and may provide status updates to the driver, such as to let the driver know that the baby is sleeping, waking up, or choking on a piece of food.

In one or more implementations, the radar-based object detection system may also include radar-emitting elements and antennas positioned on the exterior of the vehicle in order to sense and detect various external objects, such as pedestrians, other vehicles, debris or foreign objects in the road, objects on the side of the road, and so forth. The detection of such external objects may be used as part of the autonomous driving experience, as part of a “cruise control” experience, or in order to provide a warning (or automatic braking) when objects are detected in close proximity to the driving path (e.g., when a pedestrian steps into the road in front of the vehicle).

In one or more implementations, the radar-based object detection component is further configured to augment a keyless entry systems to verify that a person is actually present, or as part of an authentication procedure to authenticate the driver as a known person permitted to drive the vehicle. For example, the component may prevent the car from being driven unless the driver is authenticated as a known person permitted to drive the vehicle. In order to authenticate the driver, the component may detect biometric characteristics of the driver (e.g., a height or skeletal structure) and/or a series of “in air” gestures corresponding to a specific authentication sequence.

Example Environment

FIG. 1 is an illustration of an environment 100 in an example implementation that is operable to employ radar-based object detection in a vehicle. The illustrated environment 100 includes a vehicle computing system 102, which is configurable in a variety of ways and includes a radar-based object detection component 104, a radar module 106, and a vehicle controller 108. The vehicle computing system 102 may be incorporated as part of a vehicle 110, which in this example is illustrated as an automobile, but may include any type of vehicle such as a boat, plane, train, and so forth.

As used herein, the term “automobile” refers to a passenger vehicle designed for operation on roads and having one or more engines used rotate to wheels causing the automobile to be propelled. Examples of automobiles include cars, trucks, sport utility vehicles, vans, and the like. In one or more implementations, vehicle 110 is “autonomous” or at least “partially autonomous”. Autonomous vehicles are developed to navigate and operate either unmanned or to assist a vehicle operator, and can utilize many different types of sensors, automation, robotics, and other computer-controlled systems and mechanisms.

In this example, radar-based object detection component 104 is a hardware component of vehicle computing system 102. The radar-based object detection component 104 is configurable to detect objects in three dimensions, such as to identify the object, an orientation of the object, and/or movement of the object.

In order to detect object characteristics, radar-based object detection system 104 includes one or more radar sensors 112. Generally, radar sensors 112 include one or more antennas that are configured to transmit one or more RF signals. As a transmitted signal reaches an object (e.g., the driver of vehicle 110), at least a portion reflects back to the radar sensor 112 and is processed, as further described below, in order to detect characteristics of the object. In some cases, each radar sensor 112 includes a radar-emitting element configured to transmit the RF signal, and an antenna configured to capture the reflections of the RF signal. However, the radar sensor can include different combinations of radar-emitting elements and antennas. For instance, a single antenna could be utilized to capture reflections from three different radar-emitting elements, or vice versa. The RF signals can have any suitable combination of energy level, carrier frequency, burst periodicity, pulse width, modulation type, waveform, phase relationship, and so forth. In some cases, some or all of the respective signals transmitted in the RF signals differs from one another to create a specific diversity scheme, such as a time diversity scheme that transmits multiple versions of a same signal at different points in time, a frequency diversity scheme that transmits signals using several different frequency channels, a space diversity scheme that transmits signals over different propagation paths, etc.

Radar-based object detection component 104 can be implemented with just one, or multiple radar sensors 112. For example, in some cases, a single radar sensor 112 can be positioned proximate the driver's seat of the vehicle 110 (e.g., on the steering wheel or dashboard) in order to capture characteristics of the driver of vehicle 110. In other cases, multiple radar sensors 112 can be positioned throughout the interior of vehicle 110 in order to detect characteristics of the driver, as well as other passengers within the vehicle. In this example, radar sensors 112 are also shown as being positioned on the exterior of vehicle 110 in order to detect characteristics of objects external to vehicle 110, such as pedestrians, other vehicles, or foreign objects (e.g., trees, buildings, or debris in the road). For example, radar sensors 112 could be positioned on the front of the exterior of the vehicle in order to detect characteristics of objects within the path of the moving vehicle, as well as on the sides and rear of the exterior of vehicle 110.

The radar module 106 is representative of functionality to detect the presence or activity of persons within the vehicle 110, or objects external to the vehicle 110, and to initiate various operations based on the detection. For example, the radar module 106 may receive inputs from the radar sensors 112 that are usable to detect characteristics or attributes to identify an object (e.g., the driver of vehicle 110, a passenger, or objects located outside of the vehicle), orientation of the object, and/or movement of the object. Based on recognition of a combination of one or more of the characteristics or attributes, the radar module 106 may initiate an operation.

When radar sensors 112 are positioned proximate to the driver of vehicle 110, the radar-based object detection component 104 can monitor a presence and attention level of a driver of vehicle 110 while the vehicle is moving by initiating transmission of an outgoing RF signal via the radar sensors 112 (e.g., via a radar-emitting element), and receiving, via the radar sensor 112 (e.g., via an antenna), an incoming RF signal generated by the outgoing RF signal reflecting off the driver of the vehicle 110. Radar-based object detection component 104 can then analyze the incoming RF signal to detect one or more characteristics of the driver. Such characteristics, for example, can include a position and movement of the driver's body or a specific body part, such as the driver's hands, mouth, eyes, and so forth.

Then, based on the detected characteristics, the radar module 106 can determine an activity of the driver. As described herein, an activity of the driver corresponds to an activity currently being performed by the driver, such as driving with one or both hands on the steering wheel, being awake, drowsy, or asleep, interacting with a mobile device (e.g., texting), looking straight ahead, sideways, or backwards, talking (e.g., talking with a passenger in the vehicle or talking during a phone call), sitting somewhere other than the driver's seat (e.g., in an autonomous driving experience), or looking in the glove compartment, to name just a few.

Based on the determined activity, the radar module 106 initiates one or more operations. In some cases, the one or more operations may be initiated by sending control signals to the vehicle controller 108, in order to cause the vehicle controller 108 to control the vehicle 110 to output audible warnings, visual notifications, control navigation of the vehicle (e.g., causing the vehicle to slow down or speed up), and so forth. For example, the vehicle controller can control the audio system of the vehicle 100 to output an audible warning, such as a loud beep, or voice narration that instructs the driver to “pay attention” or “wake up”, or notifies the driver that “the baby is asleep”. As another example, the vehicle controller 108 can control the navigation system of the vehicle (e.g., the cruise control system of autonomous driving system) based on the driver's activity, such as by slowing down when the driver is not paying attention.

In some cases, the radar module 106 determines an attention level of the driver, based on the activity, and then initiates the one or more operations based on the determined attention level. For example, activities such as texting, looking backwards, or being drowsy or sleepy, may be indicative of a low attention level. In contrast, activities such as driving with both hands on the wheel are indicative of a high attention level. Thus, in some cases, a warning (e.g., audible, visual, or tactile) may be initiated, in response to determining that the driver has a low attention level, in order to alert the driver to pay attention to driving. In this way, the radar module 106 monitors the driver without requiring the driver's deliberate or conscious interaction with the system.

In one or more implementations, in order to determine whether the driver is paying attention, the attention level is first determined as a score, based on the various detected characteristics of the driver. Then, the attention level is compared to a threshold. If the attention level of the driver is above the threshold, then radar module 106 determines that the driver is paying attention to driving vehicle 110. Alternately, if the attention level of the driver is below the threshold, then radar module 106 determines that the driver is not paying attention.

In one or more implementations, the radar-based object detection component 104 monitors the presences and attention level of the driver in an autonomous or semi-autonomous vehicle system. Autonomous vehicles are developed to navigate and operate either unmanned or to assist a vehicle operator, and can utilize many different types of sensors, automation, robotics, and other computer-controlled systems and mechanisms. However, in many cases, the driver must still act as a “fallback mechanism” in order to handle driving duties in certain instances where the autonomous system fails or is unable to control navigation. In these cases, the radar-based object detection component 104 may monitor the driver's presence and attention level to ensure that the driver is a suitable backup in the event that the autonomous system needs to switch over to the manual system. In the event that the driver's attention is low, or the driver is not present in the driver's seat, the component may initiate various warnings to ensure that the driver is reminded that he may be needed as a fallback mechanism. Furthermore, in some cases, the radar-based object detection component may prevent a transition from an autonomous driving mode to a manual driving mode if the attention level of the driver is below the threshold indicating that the driver is not paying attention or is not present in the driver's seat.

Similar techniques may also be applied to passengers within the vehicle. For example, radar sensors 112 may be positioned in the rear of the vehicle 110 to monitor a toddler or baby in order to determine an activity of the passenger, such as sleeping, waking up, choking on a piece of food, and so forth. Then, an operation may be initiated, based on the activity of the passenger, such as by alerting the driver that the baby is asleep, waking up, or choking on a piece of food.

In one or more implementations, the radar-based object detection component 104 is further configured to augment a keyless entry system for the vehicle to verify that a person is actually present, or as part of an authentication procedure to authenticate the driver as a known person permitted to drive the vehicle 110. For example, the radar sensor 112 can detect one or more biometric characteristics of the driver, such as height, skeletal structure, and so forth. Then, radar module 106 can compare the detected biometric characteristics of the driver to stored biometric characteristics of known persons that are permitted to drive the vehicle. If the detected biometric characteristics of the driver match the stored biometric characteristics, then radar module 106 authenticates the driver as a known person permitted to drive vehicle 110. Alternately or additionally, the radar sensor 112 can detect one or more gestures performed by the driver. Then, radar module 106 can compare the detected one or more gestures to stored gestures corresponding to an authentication sequence. If the detected gestures performed by the driver match the stored gestures, then radar module 106 authenticates the drive as a known person permitted to drive vehicle 110. In some cases, a two-stage authentication process may be applied, whereby the driver is authenticated based on biometric characteristics as well as detection of one or more recognized gestures performed by the driver. Once the driver is authenticated, the driver is then permitted to drive the vehicle. Alternately, if the driver is not authenticated, the driver may be prevented from driving the vehicle.

As described herein, biometric characteristics correspond to distinctive, measurable characteristics that can be used to identify a particular known person, or a particular “type” of person (e.g., an adult versus a child). Biometric characteristics are often categorized as physiological versus behavioral characteristics. Physiological characteristics are related to the shape of the body and may include, by way of example and not limitation, height, skeletal structure, fingerprint, palm veins, face recognition, DNA, palm print, hand geometry, iris recognition, retina scent, heart conditions, and so forth. Behavioral characteristics are related to the pattern of behavior of a person, including but not limited to a walking gait, typing rhythm, and so forth.

When radar sensors 112 are implemented on the exterior of vehicle 110, the radar module 106 can be implemented to detect characteristics of various external objects, such as pedestrians, other vehicles, debris or foreign objects in the road, objects on the side of the road, and so forth. Vehicle controller 108 may then control navigation of the vehicle based on the detected characteristics of external objects. The detection of such external objects may be used as part of the autonomous driving experience, as part of a “cruise control” experience, or in order to provide a warning (or automatic braking) when objects are detected in close proximity to the driving path (e.g., when a pedestrian steps into the road in front of the vehicle). For example, radar module 106 can recognize the detected characteristics as certain objects, such as a pedestrian or a vehicle. Vehicle controller 108 may then control navigation of vehicle 110 based on the recognized objects, such as by slowing down when a pedestrian steps in front of the vehicle, speeding up when a vehicle approaches quickly from the rear, or swerving when a vehicle approaches from the side.

Having generally described an environment in which radar-based object detection for vehicles may be implemented, now consider FIG. 2, which illustrates an example implementation of vehicle computing system 102 of FIG. 1 in greater detail. As discussed above, vehicle computing system 102 represents any suitable type of computing system that is implemented within a vehicle, such as an automobile, plane, boat, and so forth.

Vehicle computing system 102 includes processor(s) 202 and computer-readable media 204. Radar module 106 and vehicle controller 108 from FIG. 1 embodied as computer-readable instructions on the computer-readable media 204 can be executed by the processor(s) 202 to invoke or interface with some or all of the functionalities described herein, such as through Application Programming Interfaces (APIs) 206.

APIs 206 provide programming access into various routines and functionality incorporated into radar-based object detection component 104. For instance, radar-based object detection component 104 can have a programmatic interface (socket connection, shared memory, read/write registers, hardware interrupts, etc.) that can be used in concert with APIs 206 to allow applications external to radar-based object detection component 104 a way to communicate or configure the component. In some embodiments, APIs 206 provide high-level access into radar-based object detection component 104 in order to abstract implementation details and/or hardware access from a calling program, request notifications related to identified events, query for results, and so forth. APIs 206 can also provide low-level access to radar-based object detection component 104, where a calling program can control direct or partial hardware configuration of radar-based object detection component 104. In some cases, APIs 206 provide programmatic access to input configuration parameters that configure transmit signals and/or select object recognition algorithms. These APIs enable programs, such as radar module 106, to incorporate the functionality provided by radar-based object detection component 104 into executable code. For instance, radar module 106 can call or invoke APIs 206 to register for, or request, an event notification when a particular object characteristic has been detected, enable or disable wireless gesture recognition in vehicle computing system 102, and so forth. At times, APIs 206 can access and/or include low level hardware drivers that interface with hardware implementations of radar-based object detection component 104. Alternately or additionally, APIs 206 can be used to access various algorithms that reside on radar-based object detection component 104 to configure algorithms, extract additional information (such as 3D tracking information, angular extent, reflectivity profiles from different aspects, correlations between transforms/features from different channels, etc.), change an operating mode of radar-based object detection component 104, and so forth.

Radar-based object detection component 104 represents functionality that wirelessly detects objects, such as a driver or passenger within vehicle 110, or objects external to vehicle 110. Radar-based object detection component 104 can be implemented as a chip embedded within vehicle computing system 102, such as a System-on-Chip (SoC). However, it is to be appreciated that radar-based object detection component 104 can be implemented in any other suitable manner, such as one or more Integrated Circuits (ICs), as a processor with embedded processor instructions or configured to access processor instructions stored in memory, as hardware with embedded firmware, a printed circuit board with various hardware components, or any combination thereof. Here, radar-based object detection component 104 includes radar-emitting element 208, antenna(s) 210, digital signal processing component 212, machine-learning component 214, and an object characteristics library 216, which can be used in concert to detect object characteristics using radar techniques.

Generally, radar-emitting element 208 is configured to provide a radar field. In some cases, the radar field is configured to at least partially reflect off a target object, such as a driver of vehicle 110, other passengers within vehicle 110, or objects external to vehicle 110. The radar field can also be configured to penetrate fabric or other obstructions and reflect from human tissue. These fabrics or obstructions can include wood, glass, plastic, cotton, wool, nylon and similar fibers, and so forth, while reflecting from human tissues, such as a person's body, or a part of the person's body, such as a hand, face, and so forth.

A radar field can be a small size, such as 0 or 1 millimeters to 1.5 meters, or an intermediate size, such as 1 to 30 meters. It is to be appreciated that these sizes are merely for discussion purposes, and that any other suitable range can be used. When the radar field has an intermediate size, radar-based object detection component 104 is configured to receive and process reflections of the radar field to detect large-body movements based on reflections from human tissue caused by body, arm, or leg movements. In this way, user actions, such as texting, reaching into a glove compartment, or talking with other passengers, may be detected. In other cases, the radar field can be configured to enable radar-based object detection component 104 to detect smaller and more precise movements, such as movement of the eyes of the driver or passenger in vehicle 110, or micro-gestures used to authenticate the driver. Radar-emitting element 208 can be configured to emit continuously modulated radiation, ultra-wideband radiation, or sub-millimeter-frequency radiation.

Antenna(s) 210 transmit and receive RF signals. In some cases, radar-emitting element 208 couples with antenna(s) 210 to transmit a radar field. As one skilled in the art will appreciate, this is achieved by converting electrical signals into electromagnetic waves for transmission, and vice versa for reception. Radar-based object detection component 104 can include any suitable number of antennas in any suitable configuration. For instance, any of the antennas can be configured as a dipole antenna, a parabolic antenna, a helical antenna, a monopole antenna, and so forth. In some embodiments, antenna(s) 210 are constructed on-chip (e.g., as part of an SoC), while in other embodiments, antenna(s) 210 are separate components, metal, hardware, etc. that attach to, or are included within, radar-based object detection component 104. An antenna can be single-purpose (e.g., a first antenna directed towards transmitting signals, a second antenna directed towards receiving signals, etc.), or multi-purpose (e.g., an antenna is directed towards transmitting and receiving signals). Thus, some embodiments utilize varying combinations of antennas, such as an embodiment that utilizes two single-purpose antennas directed towards transmission in combination with four single-purpose antennas directed towards reception. The placement, size, and/or shape of antenna(s) 210 can be chosen to enhance a specific transmission pattern or diversity scheme, such as a pattern or scheme designed to capture information about a micro-gesture performed by the hand. In some cases, the antennas can be physically separated from one another by a distance that allows radar-based object detection component 104 to collectively transmit and receive signals directed to a target object over different channels, different radio frequencies, and different distances. In some cases, antenna(s) 210 are spatially distributed to support triangulation techniques, while in others the antennas are collocated to support beamforming techniques. While not illustrated, each antenna can correspond to a respective transceiver path that physically routes and manages the outgoing signals for transmission and the incoming signals for capture and analysis.

Digital signal processing component 212 generally represents digitally capturing and processing a signal. For instance, digital signal processing component 212 samples analog RF signals received by antenna(s) 210 to generate digital samples that represents the RF signals, and then processes these samples to extract information about the target object. Alternately or additionally, digital signal processing component 212 controls the configuration of signals generated and transmitted by radar-emitting element 208 and/or antenna(s) 210, such as configuring a plurality of signals to form a specific diversity scheme like a beamforming diversity scheme. In some cases, digital signal processing component 212 receives input configuration parameters that control an RF signal's transmission parameters (e.g., frequency channel, power level, etc.), such as through APIs 206. In turn, digital signal processing component 212 modifies the RF signal based upon the input configuration parameter. At times, the signal processing functions of digital signal processing component 212 are included in a library of signal processing functions or algorithms that are also accessible and/or configurable via APIs 206. Thus, digital signal processing component 212 can be programmed or configured via APIs 206 (and a corresponding programmatic interface of radar-based gesture detection component 104) to dynamically select algorithms and/or dynamically reconfigure. Digital signal processing component 212 can be implemented in hardware, software, firmware, or any combination thereof.

Among other things, machine-learning component 214 receives information processed or extracted by digital signal processing component 212, and uses that information to classify or recognize various aspects of the target object. In some cases, machine-learning component 214 applies one or more algorithms to probabilistically determine an action of a driver or passenger based on an input signal and previously learned object characteristic features corresponding to the action. As in the case of digital signal processing component 212, machine-learning component 214 can include a library of multiple machine-learning algorithms, such as a Random Forrest algorithm, deep learning algorithms (i.e. artificial neural network algorithms, convolutional neural net algorithms, etc.), clustering algorithms, Bayesian algorithms, and so forth. Machine-learning component 214 can be trained on how to identify various object characteristics corresponding to user action using input data that consists of example user actions to learn. In turn, machine-learning component 214 uses the input data to learn what features can be attributed to a specific action. These features are then used to identify when the specific action occurs. In some embodiments, APIs 206 can be used to configure machine-learning component 214 and/or its corresponding algorithms. Thus, machine-learning component 214 can be configured via APIs 206 (and a corresponding programmatic interface of radar-based object detection component 104) to dynamically select algorithms and/or dynamically reconfigure.

Object characteristics library 216 represents data used by digital signal processing component 212 and/or machine-learning component 214 to identify a target object and/or detect known actions or gestures performed by the driver or passenger, or known external objects. For instance, object characteristics library 216 can store signal characteristics, characteristics about a target object that are discernable from a signal, or a customized machine-learning model that can be used to identify a user action, unique in-the-air gesture, a user identity, user presence, and so forth. In addition, certain data stored in object characteristics library 216 may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user's identity may be treated so that no personally identifiable information can be determined for the user, or a user's geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user.

FIG. 3 illustrates an example of RF wave propagation, and a corresponding reflected wave propagation. It is to be appreciated that the following discussion has been simplified, and is not intended to describe all technical aspects of RF wave propagation, reflected wave propagation, or detection techniques.

Environment 300a includes source device 302 and object 304. Object 304, for example, could be a driver or passenger in vehicle 110, or an external object (e.g., a pedestrian, other vehicle, or foreign object). Source device 302 includes antenna 306, which generally represents functionality configured to transmit and receive electromagnetic waves in the form of an RF signal. It is to be appreciated that antenna 306 can be coupled to a source, such as a radar-emitting element, to achieve transmission of a signal. In this example, source device 302 transmits a series of RF pulses, illustrated here as RF pulse 308a, RF pulse 308b, and RF pulse 308c. As indicated by their ordering and distance from source device 302, RF pulse 308a is transmitted first in time, followed by RF pulse 308b, and then RF pulse 308c. For discussion purposes, these RF pulses have the same pulse width, power level, and transmission periodicity between pulses, but any other suitable type of signal with alternate configurations can be transmitted without departing from the scope of the claimed subject matter.

Generally speaking, electromagnetic waves can be characterized by the frequency or wavelength of their corresponding oscillations. Being a form of electromagnetic radiation, RF signals adhere to various wave and particle properties, such as reflection. When an RF signal reaches an object, it will undergo some form of transition. Specifically, there will be some reflection off the object. Environment 300b illustrates the reflection of RF pulses 308a-308c reflecting off of object 304, where RF pulse 310a corresponds to a reflection originating from RF pulse 308a reflecting off of object 304, RF pulse 310b corresponds to a reflection originating from RF pulse 310b, and so forth. In this simple case, source device 302 and object 304 are stationary, and RF pulses 308a-308c are transmitted via a single antenna (antenna 306) over a same RF channel, and are transmitted directly towards object 304 with a perpendicular impact angle. Similarly, RF pulses 310a-310c are shown as reflecting directly back to source device 302, rather than with some angular deviation. However, as one skilled in the art will appreciate, these signals can alternately be transmitted or reflected with variations in their transmission and reflection directions based upon the configuration of source device 302, object 304, transmission parameters, variations in real-world factors, and so forth. Upon receiving and capturing RF pulses 310a-310c, source device 302 can then analyze the pulses, either individually or in combination, to identify characteristics related to object 304. For example, source device 302 can analyze all of the received RF pulses to obtain temporal information and/or spatial information about object 304. Accordingly, source device 302 can use knowledge about a transmission signal's configuration (such as pulse widths, spacing between pulses, pulse power levels, phase relationships, and so forth), and further analyze a reflected RF pulse to identify various characteristics about object 304, such as size, shape, movement speed, movement direction, surface smoothness, material composition, and so forth.

Now consider FIG. 4, which builds upon the above discussion of FIG. 3. FIG. 4 illustrates example environment 400 in which multiple antenna are used to ascertain information about a target object. Environment 400 includes source device 402 and a target object, shown here as hand 404. It is to be appreciated, however, that similar techniques may also be applied to other target objects, such as a driver or passenger in vehicle 110, or an external object. Generally speaking, source device 402 includes antennas 406a-406d to transmit and receive multiple RF signals. In some embodiments, source device 402 includes radar-based object detection component 104, and antennas 406a-406d correspond to antennas 208. While source device 402 in this example includes four antennas, it is to be appreciated that any suitable number of antennas can be used. Each antenna of antennas 406a-406d is used by source device 402 to transmit a respective RF signal (e.g., antenna 406a transmits RF signal 408a, antenna 406b transmits RF signal 408b, and so forth). As discussed above, these RF signals can be configured to form a specific transmission pattern or diversity scheme when transmitted together. For example, the configuration of RF signals 408a-408d, as well as the placement of antennas 406a-406d relative to a target object, can be based upon beamforming techniques to produce constructive interference or destructive interference patterns, or alternately configured to support triangulation techniques. At times, source device 402 configures RF signals 408a-408d based upon an expected information extraction algorithm, as further described below.

When RF signals 408a-408d reach hand 404, they generate reflected RF signals 410a-410d. Similar to the discussion of FIG. 4 above, source device 402 captures these reflected RF signals, and then analyzes them to identify various properties or characteristics of hand 404, such as a particular action or micro-gesture. For instance, in this example, RF signals 408a-408d are illustrated with the bursts of the respective signals being transmitted synchronously in time. In turn, and based upon the shape and positioning of hand 404, reflected signals 410a-410d return to source device 402 at different points in time (e.g., reflected signal 410b is received first, followed by reflected signal 410c, then reflected signal 410a, and then reflected signal 410d). Reflected signals 410a-410d can be received by source device 402 in any suitable manner. For example, antennas 406a-406d can each receive all of reflected signals 410a-410d, or receive varying subset combinations of reflected signals 410a-410d (i.e. antenna 406a receives reflected signal 410a and reflected signal 410d, antenna 406b receives reflected signal 410a, reflected signal 410b, and reflected signal 410c, etc.). Thus, each antenna can receive reflected signals generated by transmissions from another antenna. By analyzing the various return times of each reflected signal, source device 402 can determine shape and corresponding distance information associated with hand 404. When reflected pulses are analyzed over time, source device 402 can additionally discern movement. Thus, by analyzing various properties of the reflected signals, as well as the transmitted signals, various information about hand 404 can be extracted, as further described below. It is to be appreciated that the above example has been simplified for discussion purposes, and is not intended to be limiting.

As in the case of FIG. 3, FIG. 4 illustrates RF signals 408a-508d as propagating at a 90° angle from source device 402 and in phase with one another. Similarly, reflected signals 410a-510d each propagate back at a 90° angle from hand 404 and, as in the case of RF signals 408a-508d, are in phase with one another. However, as one skilled in the art will appreciate, more complex transmission signal configurations, and signal analysis on the reflected signals, can be utilized, examples of which are provided above and below. In some embodiments, RF signals 408a-508d can each be configured with different directional transmission angles, signal phases, power levels, modulation schemes, RF transmission channels, and so forth. These differences result in variations between reflected signals 410a-510d. In turn, these variations each provide different perspectives of the target object which can be combined using data fusion techniques to yield a better estimate of hand 404, how it is moving, its 3-dimentional (3D) spatial profile, a corresponding micro-gesture, etc.

Example Procedures

FIG. 5 is a flow diagram depicting a procedure 500 in an example implementation. The following discussion describes techniques that may be implemented utilizing the previously described systems and devices. Aspects of each of the procedure may be implemented in hardware, firmware, software, or a combination thereof. The procedure is shown as a set of blocks that specify operations performed by one or more devices and are not necessarily limited to the orders shown for performing the operations by the respective blocks.

Transmission of an outgoing RF signal is initiated via a radar-emitting element of a radar sensor implemented in a vehicle (block 902). For example, radar module 106 initiates transmission of an outgoing RF signal via radar-emitting element 208 of radar sensor 112 implemented in vehicle 110.

An incoming RF signal generated by the outgoing RF signal reflecting off an object is received by an antenna of the radar sensor (block 904). For example, antenna 210 of radar sensor 112 receives an incoming RF signal generated by the outgoing RF signal reflecting off an object, such as a driver or passenger within vehicle 110, or an object external to the vehicle 110.

The incoming RF signal is analyzed to detect one or more characteristics of the object (block 906). For example, radar module 106 may analyze the incoming RF signal to detect one or more characteristics of the driver, passenger, or external object.

Based on the detected one or more characteristics of the object, an operation is initiated (block 908). For example, the radar module 106 may determine an activity of the driver of the vehicle 110, based on the detected characteristics, and then initiate an operation based on the activity of the driver. Other examples include providing operations based on detected characteristics of passengers in the vehicle, authenticating the driver based on detected biometric characteristics or gestures, and controlling the vehicle based on the detection of external objects.

Example Vehicle Computing System

FIG. 6 illustrates various components of an example vehicle computing system 600 that incorporates radar-based object detection for vehicles as described with reference to FIGS. 1-5 Vehicle computing system 600 may be implemented as any type of a fixed or mobile device, in any form of a consumer, computer, portable, user, communication, phone, navigation, gaming, audio, camera, messaging, media playback, and/or other type of electronic device, such as vehicle computing system 102. In light of this, it is to be appreciated that various alternate embodiments can include additional components that are not described, or exclude components that are described, with respect to vehicle computing system 600.

Vehicle computing system 600 includes communication devices 602 that enable wired and/or wireless communication of device data 604 (e.g., received data, data that is being received, data scheduled for broadcast, data packets of the data, etc.). The device data 604 or other device content can include configuration settings of the device and/or information associated with a user of the device.

Vehicle computing system 600 also includes communication interfaces 606 that can be implemented as any one or more of a serial and/or parallel interface, a wireless interface, any type of network interface, a modem, and as any other type of communication interface. The communication interfaces 606 provide a connection and/or communication links between vehicle computing system 600 and a communication network by which other electronic, computing, and communication devices communicate data with vehicle computing system 600.

Vehicle computing system 600 includes one or more processors 608 (e.g., any of microprocessors, controllers, and the like) which process various computer-executable instructions to control the operation of vehicle computing system 600 and to implement embodiments of the techniques described herein. Alternatively or in addition, vehicle computing system 600 can be implemented with any one or combination of hardware, firmware, or fixed logic circuitry that is implemented in connection with processing and control circuits which are generally identified at 610. Although not shown, vehicle computing system 600 can include a system bus or data transfer system that couples the various components within the device. A system bus can include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures.

Vehicle computing system 600 also includes computer-readable media 612, such as one or more memory components, examples of which include random access memory (RAM), non-volatile memory (e.g., any one or more of a read-only memory (ROM), flash memory, EPROM, EEPROM, etc.), and a disk storage device. A disk storage device may be implemented as any type of magnetic or optical storage device, such as a hard disk drive, a recordable and/or rewriteable compact disc (CD), any type of a digital versatile disc (DVD), and the like.

Computer-readable media 612 provides data storage mechanisms to store the device data 604, as well as various applications 614 and any other types of information and/or data related to operational aspects of vehicle computing system 600. The applications 614 can include a device manager (e.g., a control application, software application, signal processing and control module, code that is native to a particular device, a hardware abstraction layer for a particular device, etc.). Computer-readable media 612 also includes APIs 616.

APIs 616 provide programmatic access to an authentication component, examples of which are provided above. The programmatic access can range from high-level program access that obscures underlying details of how a function is implemented, to low-level programmatic access that enables access to hardware. In some cases, APIs can be used to send input configuration parameters associated with modifying how signals are transmitted, received, and/or processed by an authentication component.

Vehicle computing system 600 also includes audio and/or video processing system 618 that processes audio data and/or passes through the audio and video data to audio system 620 and/or to display system 622 (e.g., a screen of a smart phone or camera). Audio system 620 and/or display system 622 may include any devices that process, display, and/or otherwise render audio, video, display, and/or image data. Display data and audio signals can be communicated to an audio component and/or to a display component via an RF link, S-video link, HDMI, composite video link, component video link, DVI, analog audio connection, or other similar communication link, such as media data port 624. In some implementations, audio system 620 and/or display system 622 are external components to vehicle computing system 600. Alternatively or additionally, display system 622 can be an integrated component of the example electronic device, such as part of an integrated touch interface.

Vehicle computing system 600 also includes a radar-based object detection component 626 that wirelessly identifies one or more features of a target object, such as a micro-gesture performed by a hand as further described above. Radar-based object detection component 626 can be implemented as any suitable combination of hardware, software, firmware, and so forth. In some embodiments, authentication component 626 is implemented as an SoC. Among other things, radar-based object detection component 626 includes radar-emitting element 628, antennas 630, digital signal processing component 632, machine-learning component 634, and object characteristics library 636.

Radar-emitting element 628 is configured to provide a radar field. In some cases, the radar field is configured to at least partially reflect off a target object. The radar field can also be configured to penetrate fabric or other obstructions and reflect from human tissue. These fabrics or obstructions can include wood, glass, plastic, cotton, wool, nylon and similar fibers, and so forth, while reflecting from human tissues, such as a person's hand. Radar-emitting element 628 works in concert with antennas 630 to provide the radar field.

Antenna(s) 630 transmit and receive RF signals under the control of authentication component 626. Each respective antenna of antennas 630 can correspond to a respective transceiver path internal to authentication component 626 that physical routes and manages outgoing signals for transmission and the incoming signals for capture and analysis as further described above.

Digital signal processing component 632 digitally processes RF signals received via antennas 630 to extract information about the target object. This can be high-level information that simply identifies a target object, or lower level information that identifies a particular micro-gesture performed by a hand. In some embodiments, digital signal processing component 632 additionally configures outgoing RF signals for transmission on antennas 630. Some of the information extracted by digital signal processing component 632 is used by machine-learning component 634. Digital signal processing component 632 at times includes multiple digital signal processing algorithms that can be selected or deselected for an analysis, examples of which are provided above. Thus, digital signal processing component 632 can generate key information from RF signals that can be used to determine what gesture might be occurring at any given moment. At times, an application, such those illustrated by applications 614, can configure the operating behavior of digital signal processing component 632 via APIs 616.

Machine-learning component 634 receives input data, such as a transformed raw signal or high-level information about a target object, and analyzes the input date to identify or classify various features contained within the data. As in the case above, machine-learning component 634 can include multiple machine-learning algorithms that can be selected or deselected for an analysis. Among other things, machine-learning component 634 can use the key information generated by digital signal processing component 632 to detect relationships and/or correlations between the generated key information and previously learned gestures to probabilistically decide which gesture is being performed. At times, an application, such those illustrated by applications 614, can configure the operating behavior of machine-learning component 632 via APIs 616.

Object characteristics library 636 represents data used by radar-based object detection component 626 to identify a target object and/or gestures performed by the target object. For instance, object characteristics library 216 can store signal characteristics, or characteristics about a target object that are discernable from a signal, that can be used to identify a user action, a unique in-the-air gesture, biometric characteristics, a user identity, user presence, and so forth. In addition, certain data stored in object characteristics library 636 may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user's identity may be treated so that no personally identifiable information can be determined for the user, or a user's geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user.

Although the embodiments have been described in language specific to structural features and/or methodological acts, it is to be understood that the various embodiments defined in the appended claims are not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the various embodiments.