MACHINE GUIDANCE SYSTEM AND METHOD FOR DEFINING AND MAINTAINING SLOPE AND/OR CROSS-SLOPE DURING EARTHMOVING OPERATIONS

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No. 63/671,519 (filed 15 Jul. 2024). This application is related to U.S. patent application Ser. No. ______ (Attorney Docket No. EQS-003US1; 661-0116US1); Ser. No. ______ (Attorney Docket No. EQS-003US3; 661-0116US3); and Ser. No. ______ (Attorney Docket No. EQS-003US4; 661-0116US4) (filed concurrently with this application). The entire disclosures of these applications are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to systems and methods for construction vehicle guidance, and more specifically, to techniques for defining, calculating, and maintaining precise slope and cross-slope parameters during grading, excavation, and earthmoving operations.

BACKGROUND

Construction vehicles play an important role in modern construction, where heavy machinery such as loaders, excavators, and dozers are routinely employed for earthmoving and material handling. Construction and related fields of work involve different tasks that are performed by different types of construction vehicles. Typical construction vehicles include loaders, excavators, compactors, and other earthmoving vehicles, any combination of which may be used at a work site.

In construction and earthmoving operations, maintaining precise slopes and cross-slopes is important for tasks such as grading, excavation, and leveling. Proper slope alignment ensures the structural integrity of roads, drainage systems, embankments, and building foundations, while accurate cross-slope alignment is essential for achieving uniform surfaces, such as parking lots or garage floors. Traditionally, operators rely on manual techniques, visual estimation, or basic tools to define and maintain slopes and cross-slopes. These methods are often time-consuming, prone to errors, and require significant operator skill and experience. Additionally, manual approaches may struggle to adapt to dynamic terrain changes or varying attachment configurations, leading to inconsistent results and reduced efficiency.

Modern construction vehicles have begun integrating guidance systems to assist operators in maintaining slopes and cross-slopes. Many of these systems, however, face limitations. Conventional systems often require complex calibration processes or rely on single-sensor setups, which may not provide the accuracy needed for precise slope alignment. Furthermore, these systems typically lack real-time feedback mechanisms, making it difficult for operators to adjust attachments or vehicle movement during operation. Environmental factors, such as uneven terrain, dust, or debris, further complicate the ability to achieve consistent slope and cross-slope alignment.

Thus, there is a need for an improved slope and cross-slope setup system that provides accurate, real-time calculations, intuitive feedback mechanisms, and automatic adjustment capabilities to ensure precise alignment of construction vehicle attachments. Such a system should be robust to environmental challenges, adaptable to dynamic terrain changes, and capable of reducing operator workload while enhancing efficiency and safety in grading, excavation, and leveling operations.

BRIEF SUMMARY OF THE DISCLOSURE

In one example, a method includes positioning an attachment of a construction vehicle at a first reference point on a terrain. The method can include obtaining location data that indicates the geographic position and elevation of that point using a location sensor. The method continues by positioning the attachment at a second reference point on the terrain. The method can include obtaining corresponding location data from the second reference point. The method further can include calculating a slope parameter based on an elevation difference. The method can include calculating this elevation difference and the horizontal distance between the two reference points. The method additionally can include generating guidance information that indicates the adjustments required to align the attachment with the calculated slope. The guidance information can be output to an operator interface for real-time feedback. The guidance information can also be output to a control system for automatic adjustment. In some examples, the method may include calculating a cross-slope parameter based on a second elevation difference. The calculation can be performed along a direction transverse to the horizontal distance by analyzing elevation differences across multiple points. The method can further include providing corresponding alignment adjustments. The method may also further include automatically adjusting the attachment using actuators. The method can include marking the reference points using physical buttons or a touchscreen interface. The method can include detecting alignment errors during operation. The method can include supplying corrective guidance based on the detected errors.

In another example, a machine guidance system includes a location sensor. The location sensor can be adapted to obtain geographic position and elevation data for at least two reference points on a terrain while onboard the construction vehicle. The system includes a processing unit that can calculate a slope parameter from the elevation difference. The processing unit can also use the horizontal distance between these reference points. The processing unit can generate guidance information to indicate the adjustments required to align an attachment of the construction vehicle with the calculated slope. The processing unit can output this guidance information to an operator interface for real-time feedback. The processing unit can also output the guidance information to a control system. The system can further calculate a cross-slope parameter based on a secondary elevation difference. The calculation may be performed by analyzing multiple points perpendicular to the horizontal distance. The system can display the guidance information in real time. The system can automatically adjust the attachment. The system can mark the reference points using onboard interfaces. The system can detect errors in slope alignment. The system can provide corrective guidance based on the detected errors.

In another example, a method includes positioning a bucket of a construction vehicle at a first reference point on a terrain. The method can include obtaining location data that includes a geographic position and an elevation using a location sensor. The method continues by positioning the bucket at a second reference point on the terrain. The method can include obtaining corresponding location data from the second reference point. The method further can include calculating a slope parameter along a defined slope direction. The processing unit can calculate the slope parameter using a first elevation difference and the horizontal distance between the reference points. The processing unit can also calculate a cross-slope parameter based on a second elevation difference. The second elevation difference is measured perpendicular to the slope direction. The processing unit can generate guidance information to indicate the adjustments required to align the bucket with both the slope parameter and the cross-slope parameter. The guidance information can be output to an operator interface for real-time feedback. The guidance information can also be output to a control system for automatic adjustment. The method can further include automatically adjusting the positions and orientation of the bucket using actuators. The method can include detecting slope alignment errors during operation. The method can include providing corrective guidance to the operator based on the detected errors.

BRIEF DESCRIPTION OF THE DRAWINGS

A detailed description of various embodiments of a machine guidance system deployed on a construction vehicle is provided below with reference to the following drawings, in which:

FIG. 1 illustrates one example embodiment of a machine guidance system;

FIG. 2 illustrates a flowchart of one example of a method for tracking a moveable part of an asset in relation to the surrounding terrain;

FIG. 3 illustrates a flowchart of one example of a method for calculating slope and/or cross-slope of a terrain surface;

FIG. 4 illustrates a perspective view of a terrain surface;

FIG. 5 illustrates an elevational view of the terrain surface;

FIG. 6 illustrates a plan view of the terrain surface;

FIG. 7 is a perspective view of one example of the machine guidance system shown in FIG. 1 onboard an asset;

FIG. 8 is a top plan view of the machine guidance system and the asset shown in FIG. 7; and

FIG. 9 illustrates a perspective view of a machine guidance assembly.

DETAILED DESCRIPTION OF EMBODIMENTS

The present detailed description provides various embodiments of a system and method for tracking construction vehicles and their surrounding terrain during operation. The described technology generally pertains to machine guidance systems for construction vehicles, including loaders, excavators, compactors, and other earthmoving vehicles, and focuses on techniques for defining, calculating, and maintaining precise slope and cross-slope parameters. While specific examples and configurations are described herein, it is important to note that these are provided for illustrative purposes only and are not intended to limit the scope of the described technology unless explicitly stated otherwise.

In construction vehicle operations, maintaining precise alignment of attachments, such as buckets or blades, with desired slope and cross-slope parameters can be important for tasks like grading, excavation, and leveling. Conventional systems often rely on manual techniques, visual estimation, or basic tools, which are prone to errors, demand significant operator skill, and struggle to adapt to dynamic terrain changes. While some modern systems integrate guidance technologies, they frequently encounter challenges such as complex calibration processes, reliance on single-sensor setups, and lack of real-time feedback mechanisms. Additionally, environmental factors like uneven terrain, dust, and debris further degrade the accuracy and reliability of these systems, making them less effective for achieving consistent results in harsh construction environments.

The present machine guidance systems and methods address these limitations by introducing a machine guidance framework that combines advanced sensor fusion, real-time terrain mapping, and intuitive operator feedback mechanisms. The framework leverages a suite of sensors, such as optical sensors (e.g., light detection and ranging, or LiDAR), location sensors (e.g., global navigation satellite system, or GNSS, receivers and antennas), and/or movement sensors (e.g., inertial movement unit, or IMU) to track the position and orientation of moveable parts of the construction vehicle, such as arms and attachments, with high precision. By utilizing reflectors placed on these moveable parts, the systems and methods generate robust point cloud data that is processed to determine attachment positions and orientations, even in dusty or debris-filled environments. The integration of specialized algorithms, such as box filters and centroid calculations, ensures that the framework can filter out noise and focus on relevant data points, enabling accurate tracking and mapping. The systems and methods support automatic calibration and setup processes, allowing operators to quickly define slope and cross-slope parameters by marking reference points on the terrain, thereby eliminating the need for complex manual calculations.

By combining advanced sensor technologies and real-time data processing, the described system significantly improves upon prior approaches, offering a robust, adaptable, and efficient solution for maintaining precise slope and cross-slope alignment in construction vehicle operations. This development not only enhances accuracy and consistency but also reduces operator fatigue and increases overall productivity in challenging worksite conditions.

While various examples of a machine guidance system deployed on a construction vehicle are described herein, not all embodiments of the inventive subject matter are limited to the specific configuration or methodologies of any of these embodiments unless explicitly recited or stated. Additionally, although the examples are described as embodying several different inventive features, any one of these features could be implemented without the others and that the inventive subject is not limited to any particular combination of features unless explicitly recited or stated.

The construction industry depends extensively on vehicles such as loaders, excavators, and dozers to carry out a variety of tasks, including earthmoving, grading, and material handling. These vehicles often incorporate moveable components, such as lift arms and attachments, which demand precise control and monitoring to maintain operational efficiency and safety. Traditional machine guidance systems, while providing some degree of assistance, face notable challenges. Many utilize wired components, which can be susceptible to failure in demanding construction environments characterized by vibration, dust, and debris. Moreover, these systems frequently struggle to adapt seamlessly to different attachments, necessitating time-consuming recalibration or manual adjustments when switching between tools. Additionally, conventional systems generally emphasize providing positional data for the vehicle or its attachments but often overlook the surrounding terrain or obstacles, limiting operators' situational awareness and increasing the likelihood of errors or accidents.

The described technology addresses these shortcomings by introducing an advanced machine guidance system that integrates multiple sensing technologies, data fusion algorithms, and real-time terrain mapping capabilities. A combination of optical sensors (e.g., LiDAR), location sensors (e.g., GNSS antennas and receivers), and movement sensors (e.g., inertial measurement units (IMUs)) is used to track the position and orientation of moveable parts of a construction vehicle with high precision. Unlike some known systems, the described technology eliminates the need for vulnerable wired components by utilizing passive reflectors on the vehicle's attachments, which reflect light pulses emitted by the optical sensors. This design enhances durability and simplifies the process of switching attachments, the machine guidance system can be efficiently recalibrated by placing reflectors on new tools and performing reduced setup steps.

The system further distinguishes itself through the capability to generate real-time terrain maps and identify obstacles in the vehicle's environment. By fusing point cloud data from the optical sensors with location and movement data, the processing unit creates a comprehensive spatial model that incorporates both the vehicle's position and the surrounding terrain. Advanced filtering techniques, such as box filters and reflectivity-based thresholds, can be used to ensure that the system remains robust even in dusty or debris-filled environments. The terrain mapping functionality enables operators to visualize the vehicle's position in relation to the terrain and obstacles, enhancing situational awareness and supporting safer, more efficient operations. Additionally, the modular architecture of the system allows deployment across a wide range of construction vehicles, including loaders and excavators, and supports both manual and autonomous operation modes.

In summary, the inventive machine guidance system overcomes the limitations of traditional approaches by combining durable hardware configurations, advanced sensor fusion, and real-time environmental awareness. This integrated solution not only enhances the precision and reliability of vehicle guidance but also provides operators with actionable insights into their surroundings, enabling safer and more efficient construction workflows.

The subject matter described herein relates to machine guidance systems deployed on assets such as construction vehicles. In some examples, the machine guidance system is used to track one or more than one moveable part of the asset and generate guidance information to assist in operation of the asset. In some embodiments, the machine guidance system or method is also used to generate a map of an area of terrain surrounding the asset and generate terrain mapping information to enable display of the asset in relation to the surrounding terrain. A variety of different types of assets may be operated using the machine guidance system, such as loaders (for example, track loaders), excavators, compactors, backhoes, dozers, etc. Other types of assets that may be operated using the machine guidance system will be apparent to one of ordinary skill in the art.

The assets may be manually operated, autonomously operated, semi-autonomously operated, or may alternate between autonomous operation mode and manual operation mode. The guidance and/or terrain mapping information is provided to an operator of the asset via a human machine interface (HMI) as part of the guidance system. The guidance and/or terrain mapping information can be provided to a control system that is deployed within the asset or that is remote from the asset. This can allow for the asset to be remotely monitored and/or controlled from afar.

I. Machine Guidance System

The machine guidance system disclosed herein may be deployed on a variety of different types of assets. The machine guidance system includes a processing unit configured to process sensor data provided by a sensor suite. The processing unit uses the processed sensor data to track the position of one or more than one moveable part of an asset. The processing unit can generate a map of an area of terrain surrounding the asset. The sensor suite may include one or more than one sensors, such as an optical sensor detecting and tracking one or more than one moveable parts of the asset. The sensor suite can include a position sensor such as a global navigation satellite system (GNSS) receiver (e.g., a global positioning system (GPS) navigation system) for determining the position of a GNSS antenna mounted on the asset. The sensor suite can include a movement sensor that determines acceleration (e.g., linear acceleration), angular velocity, and/or heading or orientation. One example of such as sensor is an inertial measurement unit (IMU) sensor for providing data relating to the rotation of the asset. The sensor suite can include a single sensor or multiple sensors. With respect to multiple sensors, the sensor suite can include at least one of each of two or more different sensors, or may include multiple sensors but less than all of the sensors described herein.

FIG. 1 illustrates one example embodiment of a machine guidance system 100. The machine guidance system 100 includes a processing unit 110. The processing unit 110 represents hardware circuitry that includes and/or is connected with one or more than one processors (e.g., integrated circuits, application-specific integrated circuits, field programmable gate arrays, graphics processing units, etc.) that perform the operations described in connection with the processing unit 110. If the processing unit 110 includes multiple processors, then the actions or operations performed by the processing unit 110 may be performed by each of the processors, or different processors may perform different actions or operations.

The processing unit 110 interfaces with the communication device 170 for communication with an off-board computing device over one or more computerized communication networks (e.g., a cellular network, a WiFi network, etc. This computing device can be a mobile phone, tablet computer, laptop computer, or the like, which may be used by an operator to input set-up information. The set-up information may include the type of asset and attachment to be operated with the assistance of the machine guidance system 100 and, in some embodiments, may also include one or more operating parameters to be used by the machine guidance system 100.

The processing unit 110 communicates with the onboard computing device 175 that can be located within the cab of the asset. The computing device 175 can be a mobile phone, a tablet computer, a laptop computer, or the like. The computing device 175 may display various guidance information and maps that can be viewed by the operator during operation of the asset

a. Optical Sensors

The machine guidance system 100 also includes one or more than one optical sensor 120, 130. The optical sensors 120, 130 track moveable parts, map terrain, and/or detect obstacles. These optical sensors 120, 130 can incudes LiDAR sensors, but in some embodiments can include stereo cameras, monocular cameras, time-of-flight (ToF) cameras, infrared (IR) sensors, radar sensors, or the like. The machine guidance system 100 includes one or more than one position sensors such as Global Navigation Satellite System (GNSS) receivers 140, 150 each respectively connected to a GNSS antenna 145, 155. The machine guidance system 100 includes a movement sensor 160, such as an inertial measurement unit (IMU) sensor. The movement sensor 160 may be included or onboard the processing unit 110, or may be separate from (but communicate with) the processing unit 110. The machine guidance system 100 in some embodiments includes a communication network device 165 that serves as a switch or connection between multiple computing devices. One example of such a network device 165 includes an Ethernet switch. The guidance system 100 can include a communication device 170 (e.g., a cellular or WiFi antenna), an onboard computing device 175, a vehicle control unit (VCU) 180 connected to an input device 182 and output devices 184, 186, and a power splitter 190 connected to a power adapter 195. The VCU 180 optionally can be referred to as an asset control unit (ACU) 180. The asset on which the machine guidance system 100 is deployed may be manufactured with these components, or one or more of these components may be later added to the asset (e.g., via upfitting). The output devices 184, 186 can be used to provide guidance information to the operator visually, audibly, tacitly and/or otherwise during operation of the asset. The aforementioned components may be separate components, or two or more (or all) of these components may be included in a single device.

The optical sensors 120, 130 are mounted at different location on the asset. In some embodiments, a single optical sensor 120 or 130 may be used, or more than two optical sensors 120, 130 may be used. One optical sensor 120 or 130 may be used to reduce the overall production cost of the machine guidance system 100, while three or more optical sensors 120, 130 may be used to provide redundancy. For example, a third or fourth (or more) optical sensor 120 and/or 130 may be included in the machine guidance system 100. If an optical sensor 120 or 130 fails, then the third or fourth (or other) optical sensor may be used in place of the failed optical sensor 120 or 130. As another example, data may be received from each of the three or more optical sensors 120, 130 and used for redundancy purposes. Two optical sensors 120, 130 are used in the illustrated example to provide a continuous 360 degree view around the asset (without the asset having to turn or rotate to provide the 360 degree view).

Each of the optical sensors 120, 130 can generate optical data representative of objects (or the absence of objects) within fields of view of the optical sensors 120, 130. With respect to LiDAR sensors, the optical sensors 120, 130 generate point cloud data based on light pulses emitted and reflected back from moveable parts of the asset. For example, the asset may include reflective surfaces or reflective objects (e.g., stickers, panels, etc.) may be affixed to the moveable parts of the asset. For example, if the construction vehicle is a loader, a reflector may be placed on the lift arm of the loader and a reflector may be placed on an attachment attached to the lift arm of the loader. The reflectors may be passive devices that reflect light back to the optical sensors 120, 130. For example, the reflectors may be retroreflectors that are not powered, and do not require power (electrical or otherwise), to operate. The reflectors may not be wired or otherwise conductively coupled with any power source. The reflectors may be tape, sheeting or other material with a reflective surface that is suitable for reflecting light back to a LiDAR sensor, such as the white aluminum foil tape. However, other optical sensors 120, 130 may be used, and other reflectors or no reflectors may be used.

The non-wired reflective surfaces (e.g., tape or plates) placed on moveable parts of the asset eliminates vulnerabilities or failure points associated with wired systems, such as damage from vibration, dust, or debris. This can help the machine guidance system 100 operate in harsh construction environments. In general, a harsh construction environment is a worksite characterized by challenging or extreme physical conditions that can impact the safety of workers, the durability of materials, and the overall progress and success of the project.

b. Asset Control Unit (ACU)

The ACU 180 represents the central processing and control module of the asset. The ACU 180 represents hardware circuitry that includes and/or is connected with one or more processors (e.g., one or more integrated circuits, application-specific integrated circuits, field programmable gate arrays, etc.) that perform the operations described herein in connection with the ACU 180. The processing unit 110 communicates with the ACU 180 of the asset. The asset control unit 180 is the central processing and control system integrated into the asset. The asset control unit 180 serves as the primary interface between the hardware components of the asset (e.g., sensors, actuators, and attachments) and the operator. The asset control unit 180 optionally can autonomously control operation of the asset based on data and signals provided by the processing unit 110.

The ACU 180 receives input from the operator via input devices 182 and/or from the processing unit 110, with this input directing changes in movement of the asset, positions of arms of the asset, and/or positions/orientations of the attachment. For example, the ACU 180 can control cylinders, motors, engines, or the like, to move the asset, asset arms, and/or asset attachment based on input from the processing unit 110 and/or operator.

The ACU 180 or the processing unit 110 actively controls or limits the movement of the asset and the attachments of the asset based on, among other information, the slope parameter and/or the cross-slope parameter that are measured as described herein. For example, the ACU 180 and/or processing unit 110 can use the calculated slope and/or cross-slope parameters to dynamically control or adjust the operation of the asset to precisely align attachments, such as blades or buckets, with the desired terrain or earthmoving specifications. This control is implemented using real-time feedback, automatic adjustments, and/or operator guidance. The guidance system 100 includes output devices 184, 186 that provide real-time visual feedback to the operator via an interface, such as a touchscreen display or configurable LED light bars. These indicators show whether the attachment is aligned with the desired slope and cross-slope parameters.

For example, the output devices 184, 186 may display green lights or positive indicators to signal proper alignment to the operator. The output devices 184, 186 can display red lights or warnings to indicate deviations from the desired slope or cross-slope to the operator. Additionally or alternatively, the output devices 184, 186 may include audio output devices (e.g., speakers) that generate audio cues to alert the operator when the attachment deviates from the desired slope and/or cross-slope parameters, prompting corrective action. The asset may include hydraulic actuators that are automatically adjusted by the ACU 180 and/or processing unit 110 to control the position and orientation of the attachment (e.g., blade or bucket) based on the calculated slope and/or cross-slope parameters. These adjustments can include tilting the attachment to match the slope angle or to match a desired slope angle to move materials and generate a desired slope, leveling the attachment to maintain the desired cross-slope, etc. As the vehicle moves across the terrain, the processing unit 110 can continuously or repeatedly update the slope and/or cross-slope calculations, and adjust the attachment in real time to maintain alignment based on the updated slope and/or cross-slope calculations.

The ACU 180 and/or processing unit 110 may limit or adjust the speed at which the construction asset is moving based on the slope and/or cross-slope parameters to ensure safe operation, especially on steep inclines or uneven terrain. The ACU 180 and/or processing unit 110 can modify the trajectory of the asset to follow the desired slope or cross-slope path. This provides for consistent grading or excavation along the defined parameters. The processing unit 110 can continuously or repeatedly monitor the alignment of the attachment to the desired slope and/or cross-slope parameters, and also detect deviations from the desired slope or cross-slope parameters. If deviations are detected, the ACU 180 and/or processing unit 110 provides corrective guidance to the operator or automatically adjusts the attachment to realign the attachment with the calculated parameters.

The processing unit 110 and/or ACU 180 calculates the slope and/or cross-slope parameters, and controls or directs control of the asset based on these parameters, to control the blade or bucket to maintain the desired slope and cross-slope for roads, embankments, or drainage systems. The processing unit 110 and/or ACU 180 automatically controls or directs manual control of the depth and angle of excavation by controlling or directing manual control of positions and orientation of the attachment to maintain consistent slopes for trenches or other earthmoving tasks. The processing unit 110 and/or ACU 180 automatically adjust or direct manual adjustment of the attachment to achieve uniform cross-slopes for flat surfaces, such as parking lots or building foundations.

For example, the processing unit 110 can calculate the slope and/or cross-slope parameters based on data from the GNSS receivers 140, 150 and GNSS antennas 145, 155, the movement sensor 160, and/or optical sensors 120, 130, as described herein. The processing unit 110 can generate visual and/or audio feedback for the operator of the asset using the output devices 184, 186 and/or the onboard computing device 175 to indicate alignment status of the attachment. The alignment status of the attachment is the real-time condition indicating how closely the position and orientation of the attachment-such as a blade, bucket, or other earthmoving tool-matches the desired slope and cross-slope parameters set for the current operation and location of the asset. This alignment status is determined by the processing unit 110 comparing the actual spatial configuration of the attachment, as calculated from sensor data (including the GNSS receivers 140, 150, the GNSS antennas 145, 155, the movement sensor 160, and/or the optical sensors 120, 130), to the target slope and cross-slope values defined by the project requirements or operator input. The alignment status may be visually represented to the operator through a user interface, such as a display or LED light bar of the output devices 184, 186 and/or the onboard computing device 175, using color codes or graphical indicators to show whether the attachment is on grade, above, or below the intended alignment. A “good” or “on-grade” alignment status means the attachment is properly positioned to achieve the specified terrain profile, while a “misaligned” status signals that corrective action-either manual or automatic by the processing unit 110 or ACU 180—is needed to bring the attachment back into compliance with the desired parameters. This feedback provides for precise, efficient, and safe grading, excavation, or leveling operations.

Actuators, such as hydraulic cylinders, pneumatic cylinders, electric motors, or the like, onboard the asset are controlled by the processing unit 110 and/or ACU 180 adjust the tilt and/or position of the attachment to align the attachment (e.g., the cutting edge of a bucket) with the calculated slope and/or cross-slope parameters. The processing unit and/or ACU 180 can modify the moving speed and trajectory of the asset to maintain consistent operation of the attachment along the slope and cross-slope.

c. Location Sensors

The location sensors (e.g., the GNSS antennas 145, 155 and associated receivers 140, 150) can be mounted at different locations on the asset. Each of the GNSS antennas 145, 155 receives and, in some embodiments, amplifies signals transmitted or broadcast by GNSS satellites and converts the signals for use by the GNSS receivers 140, 150. The GNSS receivers 140, 150 analyze the received signals to determine the positions of the GNSS antennas 145, 155. In one example, one GNSS antenna 145 serves as a system reference point for the machine guidance system 100, and another GNSS antenna 155 is used to determine heading and/or pitch of the asset. The GNSS receivers 140, 150 use real-time kinematics (RTK) positioning technology to provide more precise position information in one example.

The machine guidance system 100 may include two GNSS antennas 145, 155 mounted on the asset or may include more than two GNSS antennas 145, 155. The GNSS antennas 145, 155 and GNSS receivers 140, 150 work together to provide precise positional, heading, and pitch information for the asset. The GNSS antennas 145, 155 can be placed on the asset to serve distinct but complementary purposes. One GNSS antenna 145 can be placed closer to a front or leading edge of the asset and operate as the primary reference point for the machine guidance system 100. The signals received by this front GNSS antenna 145 are examined by the GNSS receiver 140 to provides the absolute position of the asset (e.g., in a global coordinate system, such as latitude, longitude, and altitude. This GNSS antenna 145 serves as the fixed reference for calculating heading and pitch when combined with data from the rear GNSS antenna 155. The front GNSS antenna 145 can be mounted on a stable, fixed part of the asset, typically near the front or center of the body of the asset.

The other GNSS antenna 155 can be mounted on the asset farther from the front than the front GNSS antenna 145 (and closer to the opposite back of the asset than the front GNSS antenna 145). The rear GNSS antenna 155 can be mounted on a moveable or rear part of the asset, such as the rear linkage or a stable rear section of the asset. The rear GNSS antenna 155 works with the front GNSS antenna 145 to calculate heading and pitch of the asset. The rear GNSS antenna 155 measures the relative position of the rear of the asset compared to the front. The rear GNSS antenna 155 provides signals to the GNSS receiver 150, which uses the signals to calculate the heading (e.g., the direction of travel) of the asset by calculating the angle between the two GNSS antennas 145, 155. The GNSS receiver 150 also can calculate the pitch of the asset (e.g., the tilt of the asset along its longitudinal axis) by comparing the vertical displacement between the two GNSS antennas 145, 155.

In some embodiments, the machine guidance system 100 does not include the GNSS receivers 140, 150 and/or antennas 145, 155. In these embodiments, the machine guidance system 100 uses other techniques to determine the real-world geographic position of the asset. For example, the machine guidance system 100 can include one or more than one reflector positioned at a known location. The optical sensor 120 and/or the optical sensor 130 generate point cloud data based on the light pulses emitted and reflected back from that reflector to determine the position of the optical sensor 120 and/or the optical sensor 130 and, therefore, the location of the asset relative to the reflector. Because the reflector location is known, the location of the asset relative to the reflector can then be converted into the location of the asset.

d. Movement Sensor

The movement sensor 160 is mounted on the asset and provides data relating to movement of the asset, such as rotation of the asset. This data can be repeatedly provided by the movement sensor 160 (e.g., to the processing unit 110), such as in a continuous stream or otherwise repeated stream of data. The movement sensor 160 can output signals indicative of roll, pitch, and/or yaw rotation of the asset. The movement sensor 160 can be mounted at or close to the center of rotation of the asset, or in another location.

e. Processing Unit

The processing unit 110 receives sensor data from the sensors of the machine guidance system 100 via the network device 165. The network device 165 (e.g., an Ethernet switch) manages the flow of sensor data from the optical sensors 120, 130, the movement sensor 160, and/or the location sensors (e.g., the receivers 140, 150) to the processing unit 110. The network device 165 may be a ruggedized gigabit Ethernet switch, although other components capable of performing packet switching (e.g., in accordance with the Ethernet or Industrial Ethernet (IE) standard) may be used.

1. Data Fusion

The processing unit 110 fuses or otherwise combines the sensor data received from the sensors in the sensor suite (e.g., the optical sensors 120, 130, the GNSS receivers 140, 150, the GNSS antennas 145, 155, and/or the movement sensor 160). The processing unit 110 uses the fused data to track moveable parts of the asset in relation to the surrounding terrain. The processing unit 110 uses the fused data to provide guidance and/or terrain mapping information to an operator of the asset.

The GNSS receivers 140, 150 provide real-time positional data for the asset. This allows for the processing unit 110 to precisely track the location of the asset at a worksite. This is useful for tasks such as mapping the terrain, defining work boundaries, and ensuring accurate excavation or grading. The heading of the asset can be used by the processing unit 110 to maintain alignment of the asset during operations such as trenching, grading, or material placement. The pitch of the asset can be used by the processing unit 110 to maintain proper blade or bucket angles, and ensure accurate grade control. The GNSS data can be fused with data from other sensors, such as the optical sensors 120, 130 and/or the movement sensor 160, to comprehensively track the position, orientation, and movement of the asset. This fusion improves the accuracy and reliability of the machine guidance system 100, especially during operation on dynamic or uneven terrain.

Using multiple GNSS antennas 145, 155 can provide more precise heading and pitch calculations compared to a single GNSS device. The combination of absolute positioning (e.g., using the front GNSS antenna 145) and relative positioning (e.g., using the rear GNSS antenna 155) allows for terrain mapping, grade control, obstacle avoidance, and the like.

Using the data output by dual or multiple GNSS antennas 145, 155 (e.g., front and rear antennas) in combination with the data output by the optical sensors 120, 130 can provide more precise positional, heading, and pitch information for the asset when compared with other machine guidance systems. For example, a dual location sensor configuration enables real-time tracking of the orientation and movement of the asset, which can be helpful for tasks such as grade control and terrain mapping. The dual location sensor setup provides increased accuracy for heading and pitch calculations compared to machine guidance systems that rely on single location sensors, while the integration with the optical sensors 120, 130 improves terrain mapping and obstacle detection.

2. Communication with Computing Devices

The processing unit 110 interfaces with the communication device 170 for communication with an off-board computing device over one or more than one computerized communication networks (e.g., a cellular network, a WiFi network, etc. This computing device can be a mobile phone, tablet computer, laptop computer, or the like, which may be used by an operator to input set-up information. The set-up information may include the type of asset and attachment to be operated with the assistance of the machine guidance system 100 and, in some embodiments, may also include one or more than one operating parameters to be used by the machine guidance system 100.

The processing unit 110 communicates with the onboard computing device 175 that can be located within the cab of the asset or on the roof of the asset. The onboard computing device 175 can be a mobile phone, a tablet computer, a laptop computer, or the like. The onboard computing device 175 may display various guidance information and maps that can be viewed by the operator during operation of the asset.

f. Input Devices and Output Devices

The processing unit 110 communicates with the asset control unit 180 of the asset. The asset control unit 180 is the central processing and control system integrated into the asset. The asset control unit 180 serves as the primary interface between the hardware components of the asset (e.g., sensors, actuators, and attachments) and the operator. The asset control unit 180 can, in some embodiments, autonomously and/or semi-autonomously control operation of the asset based on data and signals provided by the processing unit 110.

The asset control unit 180 controls actuators that adjust the position and movement of asset components, such as the position of a bucket or blade of the asset to provide proper alignment and grade control, the movement of arms of the asset, hydraulic pressures of the asset for control of the arms and attachments, and the like. The asset control unit 180 provides an operator interface and can output visual and/or audio feedback to the operator via displays, light bars, etc.

The asset control unit 180 interfaces with one or more than one input device 182 and one or more than one output device 184, 186, which may be located within the cab of the asset. The input device 182 may be a button, switch, lever, selectable icon on a graphical user interface, etc. The input device 182 can be used by the operator to input information during the set-up process. In another example, the operator may input this information using the onboard computing device 175. The output devices 184, 186 visually convey positional feedback information to the operator. For example, the output devices 184, 186 may be elongated displays or lamps (e.g., light bars) that illuminate to communicate positions of the asset, portions of the asset (e.g., arms of the skid steer loader), and/or attachments (e.g., a bucket connected to the arms). The output devices 184, 186 are elongated LED light bars used to provide guidance information to the operator during operation of the asset, as described above. During the set-up process, the output devices 184, 186 may be configured to operate in different modes, such as a standard mode, a dual mode, or a quad mode, as described herein. The onboard computing device 175 also can be an input and/or output device of the machine guidance system 100.

g. Base Station

The machine guidance system 100 also includes a base station 188. The base station 188 can be located off-board the asset, and may be a stationary component of the machine guidance system 100. For example, the base station 188 can be still while the asset moves at a worksite. The base station 188 may be moveable between different worksites (e.g., upon completion of work or usage of the machine guidance system 100 at one worksite, the base station 188 can be moved to another worksite).

The base station 188 can provide a fixed, high-accuracy reference point for the machine guidance system 100, such as for the GNSS receivers 140, 150 and the GNSS antennas 145, 155. The base station 188 can include (and/or the base station 188 shown in FIG. 1 can represent) one or more GNSS antennas (e.g., similar or identical to the antenna 145 and/or 155) and/or one or more GNSS receivers (e.g., similar or identical to the receiver 140 and/or 150). The base station 188 may include or be connected to a power supply, such as a generator, power utility grid, battery or battery cells, etc. The base station 188 can include one or more than one communication device for wirelessly communicating with the processing unit 110 of the machine guidance system 100 (e.g., via the communication device 170). The communication device of the base station 188 may be similar or identical to the communication device 170. The base station 188 can include a processing unit similar or identical to the processing unit 110 of the machine guidance system 100.

The GNSS antenna of the base station 188 receives GNSS satellite signals that are used by the GNSS receiver of the base station 188 to calculate a geographic location (e.g., longitude, latitude, and/or altitude) of the base station 188. The processing unit of the base station 188 may communicate with the processing unit 110 of the machine guidance system 100, compare locations determined by the GNSS receivers 140 and/or 150 of the machine guidance system 100 and determined by the GNSS receiver of the base station 188, and decide whether the machine guidance system 100 is within a threshold distance limit from the base station 188. For example, the processing unit 110 of the machine guidance system 100 may not permit autonomous or semi-autonomous operation of the asset, terrain mapping or updating of terrain maps, etc. if the machine guidance system 100 (and, therefore, the asset) are more than five miles from each other (as one example, although other distances may be used).

The processing unit of the base station 188 can receive a designated location (e.g., longitude, latitude, and/or altitude) from an operator of the machine guidance system 100 or from another source (e.g., output from a survey of a worksite). The processing unit of the base station 188 compares this input location and compare the input location with the location calculated using the GNSS satellite signals received by the GNSS antenna of the base station 188. The processing unit of the base station 188 can calculate a difference, or error, between these locations. A correction can be calculated based on this difference, such as values to add or subtract to the longitude, latitude, and/or altitude values calculated by the GNSS receiver of the base station 188. This correction can be communicated from the base station 188 to the processing unit 110 of the machine guidance system 100. The processing unit 110 can then apply the correction to locations calculated by the GNSS receiver(s) 140, 150 of the machine guidance system 100 to ensure that any errors in the locations determined from the GNSS satellite signals are corrected.

In another example, the base station 188 may be mobile. For example, the base station 188 may include or be on wheels, tracks, or the like, for self-propelling or being moved (manually or with the aid of the asset or another vehicle). In another example, the base station 188 may be part of or coupled to a stationary object, such as a building or another structure. As another example, one of the GNSS antennas 145, 155 can receive signals for establishing the reference location described above.

II. Tracking Process

FIG. 2 illustrates a flowchart of one example of a method 200 for tracking a moveable part of an asset in relation to the surrounding terrain. While the operations of the method 200 are generally described with respect to FIG. 1, the operations may otherwise be performed. The method 200 can be used to track the cutting edge of a bucket attachment of a loader during the performance of grading (e.g., the process of shaping and leveling the ground before building). The method 200 can be used to position the cutting edge of the bucket attachment at a desired elevation relative to the elevation of the terrain. In some embodiments, the method 200 may be used in connection with other assets and/or attachments to perform different tasks.

The method 200 includes parallel processing operations—for example, the sensor data collection operations 202, 204, and/or 206 can be performed in parallel or during overlapping time periods, the sensor data processing operations 208, 210, and/or 212 can be performed in parallel or during overlapping time periods, the arm/attachment reflector and terrain detection operations 214 and 216 can be performed in parallel or during overlapping time periods, the terrain and vehicle transform and cutting edge kinematics operations 222, 220, and/or 218 can be performed in parallel or during overlapping time periods, the operations 224 and 226 relating to the display of cutting edge and terrain elevations via a user interface (e.g., the onboard computing device 175 and/or light bars 184, 186) can be performed in parallel or during overlapping time periods.

a. Data Collection and Fusion

At 202, optical data related to reflectors on the asset are collected. For example, the optical sensors 120, 130 can generate point cloud data based on light pulses emitted and reflected back from a reflector placed on a lift arm of the asset and/or another reflector placed on the attachment connected to the lift arm (e.g., the bucket attached to the lift arm). At 204, movement data representative of movement of the asset is generated. For example, the movement sensor 160 can generate movement data indicative of movement of the asset. This data can include roll, pitch, and/or yaw rotation parameters for the asset. At 206, location data is obtained and used to determine positions. For example, the GNSS receivers 140, 150 can analyze electrical signals received from the GNSS antennas 145, 155, respectively, to determine the positions of the GNSS antennas 145, 155. In some embodiments, the location sensors use RTK positioning technology to provide more precise position information about the asset (such as CEP accuracy of about one centimeter).

At 208, 210, and 212, data is fused and processed. For example, the processing unit 110 can process the point cloud data provided by the optical sensors 120, 130 (at 208), process the movement data provided by the movement sensor 160 (at 210), and process the position data provided by the GNSS receivers 140, 150 (at 212). In some embodiments, less than all of this data is processed. The data can be processed by receiving and fusing the different sensor data based on the timestamps associated with the sensor data. For example, point cloud data, IMU data, and GNSS data may be fused (e.g., combined or associated with each other) if the respective timestamps are within a specified tolerance of each other. As another example, one or more Kalman filters or complimentary filters can be used to fuse the data. If one of the sensors 120, 130, 160, GNSS antennas 145, 155, and/or GNSS receivers 140, 150 fails or generates data that is outside of an acceptable range of values, the processing unit 110 can fuse the data from the remaining sensors 120, 130, 160, GNSS antennas 145, 155, and/or GNSS receivers 140, 150 by replacing the data from the failed sensors 120, 130, 160, GNSS antennas 145, 155, and/or GNSS receivers 140, 150 with data from another one of the sensors 120, 130, 160, GNSS antennas 145, 155, and/or GNSS receivers 140, 150 that has not failed. For example, if a GNSS antenna 145, 155 or GNSS receiver 140, 150 fails, the processing unit 110 can use data from the movement sensor 160 to replace the movement, velocity, pitch, etc. data that otherwise may have been obtained by the failed GNSS antenna 145, 155 and/or GNSS receiver 140 and/or 150.

b. Reflector Position Calculations

At 214, the position of one or more than one reflector on part of the asset is or are determined. For example, the processing unit 110 can analyze the optical data (from 208) to detect the position of a first reflector placed on part of the arm of the asset and the position of a second reflector placed on the attachment that is connected with (and separately moveable from) the arm. The positions of the first and second reflectors may be detected, for example, by calculating the centroids of the reflectors from the optical data. As another example, the positions of the first and/or second reflectors may be identified by manually measuring the position(s) of the first and/or second reflectors.

The method 200 can include filtering data at 214. For example, the processing unit 110 can filter out data points having reflectivity or signal values below a predetermined threshold. This can make the method 200 and system 100 robust to dust in the environment. For example, the method 200 can disregard weak or low-quality signals that may result from environmental factors such as dust, fog, or other airborne particulates that can scatter or attenuate light signals. By applying this filtering mechanism, the method 200 ensures that only stronger, more reliable data points are used for tasks such as terrain mapping, obstacle detection, and tracking of moveable parts. For example, the reflectivity values of the data points may vary between a lower or minimum value and an upper or maximum value. The predetermined threshold used to filter out data points having lower reflectivity values may be 50% of the upper or maximum value, 60% of the upper or maximum value, or another percentage. This allows the method 200 to maintain accurate and consistent performance even in harsh or dusty environments commonly encountered on construction sites. The filtering process reduces noise in the data, improving the overall reliability and precision of the method 200.

As another example, the processing unit 110 (at 214) can use one or more than one box filter to filter out data points that are not associated with the reflectors (given the known positions of the reflectors in relation to the known position of the front antenna 145, which serves as a system reference point). The box filter can be a spatial filter that applies a uniform averaging operation over a defined region, or box, of data points. The processing unit 110 defines a rectangular or cubic region around a target data point in a dataset, such as a 3D point cloud, received from the optical sensor(s) 120, 130.

An operation such as averaging or summing is applied to these data points within the box to calculate an output value for the target point (e.g., the reflector in the point cloud). The size of the box (e.g., the width, height, and depth of the box) determines the range of data points included in the operation. The size of the box can be adjusted based on the specific requirements of the application, such as the density of the point cloud or the level of noise in the environment. The box filter smooths the point cloud data by averaging the values of points within the defined box. This helps to reduce random noise caused by environmental factors such as dust, debris, or sensor inaccuracies. The filter can exclude outlier points that deviate significantly from the surrounding data. For example, points with unusually high or low reflectivity values may be removed to improve the accuracy of terrain mapping and obstacle detection.

By aggregating data within the box, the filter reduces the overall complexity of the optical data (e.g., the point cloud). The box filter can be used by the processing unit 110 to isolate and enhance data points associated with reflectors placed on moveable parts of the asset. By focusing on points within a specific region, the processing unit 110 can more accurately track the position and orientation of arm and/or attachment of the asset.

In some examples, the processing unit 110 (at 214) compares the number of filtered data points to a number of points expected to be returned by each reflector to detect one or more than one error. For example, the processing unit 110 determines that the number of filtered data points (or the average or sum of the filtered data points) indicates an error when the number, average, or sum falls below a lower threshold. The errors that can be identified by the processing unit 110 in this way can be an object blocking the view between the optical sensor 120, 130 and the reflector, the reflector falling off the asset, damage to the reflector, a dirty optical sensor 120, 130, too much dust in the environment, a foreign reflective object close to the reflector, etc. If an error is detected, the processing unit 110 can return an error message so that the operator can identify and correct the error. In one example, the processing unit 110 may prevent continued movement of the asset, the arm, and/or the attachment responsive to such an error being identified.

c. Terrain Mapping

At 216, the optical data is analyzed to detect terrain elevation around the asset and/or generate or update a terrain map showing obstacles near the asset. The processing unit 110 analyzes the point cloud data from 208 (and which may be filtered) to detect the elevation of the terrain surrounding the asset, as well as generate a terrain map that may include the presence of any obstacles near the asset.

The processing unit 110 (at 216) can segment the point cloud to separate or differentiate terrain points from non-terrain objects, such as vehicles, trees, or buildings. These operations can be performed using algorithms that classify points based on height, reflectivity, and/or clustering. For example, the processing unit 110 differentiates between obstacles and terrain features by analyzing the point cloud data generated by the optical sensors 120, 130, along with data from other sensors like the location sensors and the movement sensor 160. The processing unit 110 differentiates between obstacles and terrain features using segmentation, classification, and filtering techniques. The point cloud data represents the surrounding environment, including both terrain features (e.g., ground, slopes) and obstacles (e.g., rocks, equipment, personnel). The location data from the location sensors and the movement data from the movement sensor 160 indicate the position and orientation of the asset, which is used by the processing unit 110 to identify the relative location of detected objects.

The processing unit 110 (at 216) can preprocess the point cloud data by applying noise filter(s) and/or outlier removal (e.g., using box filters). The processing unit 110 segments the point cloud into distinct clusters or regions to isolate potential obstacles from the terrain. The processing unit 110 can use progressive morphological filtering or cloth simulation filtering to identify ground points, or points in the data cloud indicative of the terrain. This can involve the processing unit 110 analyzing the relative height of points and their spatial distribution to distinguish ground points (e.g., terrain) from elevated objects. The non-ground points are grouped into clusters by the processing unit 110 based on the spatial proximity of the points using clustering algorithms (e.g., density-based spatial clustering of applications with noise) or k-means. Each cluster can represent a potential obstacle or terrain feature.

For each cluster, the processing unit 110 (at 216) extracts features to help classify the cluster as an obstacle or a terrain feature. These extracted features can include the height of the cluster of data points above the ground. The processing unit 110 can identify objects that are significantly elevated above the ground as obstacles (e.g., data point clusters that are at least a threshold height above the ground). The processing unit 110 can examine the dimensions (e.g., width, height, and/or depth) and shape of the clusters to differentiate between obstacles and terrain. For example, small, irregularly shaped clusters may be identified by the processing unit 110 as obstacles (e.g., rocks or debris), while larger, flatter clusters may represent terrain features (e.g., slopes or embankments). The processing unit 110 can examine reflectivity values of the data points in the cluster. The data points having greater reflectivity may be identified as metallic objects (e.g., obstacles such as other equipment), while lower reflectivity data points may be identified as the terrain. The processing unit 110 can examine the data points in the clusters to determine whether the cluster(s) is or are moving. If a cluster is detected by the processing unit 110 to be moving (e.g., using temporal data from consecutive LiDAR scans), the processing unit 110 can identify that cluster as being an obstacle (e.g., a person, animal, or vehicle).

The processing unit 110 (at 216) can classify each cluster as either an obstacle or a terrain feature using machine learning and/or rule-based algorithms. The processing unit 110 can compare the data and values to predefined thresholds for this classification. For example, predefined thresholds for features like height, size, and reflectivity are used to classify clusters. Clusters having a height above a certain threshold (e.g., 0.5 meters) are classified as obstacles, while clusters with large, flat shapes are classified as terrain features. The processing unit 110 can use supervised learning models (e.g., decision trees, support vector machines, or neural networks) trained on labeled datasets to classify clusters based on extracted features. These models can learn complex patterns and improve classification accuracy over time.

The processing unit 110 (at 216) can use temporal data from consecutive LiDAR scans to refine the classification of clusters. Static objects (e.g., rocks, terrain features) have data points that remain in the same or substantially same location across multiple scans. Conversely, dynamic objects (e.g., personnel, vehicles) may have data points that change position over time and are classified as obstacles. The processing unit 110 can compare a current point cloud with previous scans to detect newly introduced objects, which may be identified as obstacles.

The processing unit 110 (at 216) can integrate or fuse data from other sensors to improve the cluster classification. For example, the processing unit 110 can use the movement data from the movement sensor 160 to account for the roll, pitch, and/or yaw of the asset. This helps the processing unit 110 correctly identify terrain features even while the asset is on uneven ground. The processing unit 110 can use GNSS location information to differentiate between stationary obstacles and terrain features in the context of the location of the asset.

The processing unit 110 (at 216) can repeatedly update the classification of clusters as new point cloud data is collected. For example, the processing unit 110 can dynamically change the classification of an object from a terrain feature to an obstacle responsive to the cluster starting to move in successive scans.

The processing unit 110 (at 216) can generate the terrain map by converting the processed point cloud data into a structured representation. The processing unit 110 divides the terrain into a grid of cells (e.g., a 2D raster grid). For each cell, the processing unit 110 calculates an average, minimum, or maximum elevation of the data points within each cell. If no points exist in a cell, interpolation methods (e.g., nearest neighbor or bilinear interpolation) can be used by the processing unit 110 to estimate the elevation for that cell. The terrain map may be smoothed using techniques such as Gaussian filtering to reduce abrupt changes and create a more realistic representation. The terrain map can be integrated into the machine guidance system 100 to assist with path planning, grade control, and obstacle avoidance.

The processing unit 110 (at 216) can repeatedly update the terrain map as new point cloud data is collected. For example, as the asset moves, new point cloud data is merged with the existing terrain map to provide real-time updates. The processing unit 110 can detect changes in the terrain (e.g., newly excavated areas or obstacles) by comparing the updated point cloud with the existing map.

The processing unit 110 (at 216) generates the terrain map as a three-dimensional terrain map in real-time (e.g., the terrain map is generated or updated as the data is collected without introducing additional delays outside of normal computer processing). This terrain map can be created using data from optical sensors 120, 130, location data from the location sensors, and movement data from the movement sensor 160. The processing unit 110 differentiates obstacles from terrain features using these data sources.

d. Position and Orientation Calculation

At 218, the location data from the location sensors and the movement data from the movement sensor 160 is analyzed to determine the real-world position and orientation of the asset. The processing unit 110 can analyze the GNSS data from 212 and the IMU data from 210 to determine the geographic position and orientation (or heading) of the asset in a coordinate frame, such as the north-east-down (NED) coordinate frame. At 220, the processing unit 110 analyzes the terrain data from 216 and the movement data from 210 to map the terrain surrounding the asset (including any detected obstacles) in the NED coordinate frame or another coordinate system.

At 222, the reflector data from 214 is analyzed along with known dimensions of the asset to determine the position and orientation of the arm of the asset and/or of the attachment, such as the cutting edge of the bucket attachment. The processing unit 110 can determine the positions and orientations using models that mathematically describe the asset configuration through the use of forward kinematics. This determination may be performed using models that mathematically describe the asset configuration through the use of forward kinematics. Forward kinematics as used by the processing unit 110 is a mathematical modeling approach to determine the position and orientation of the end effector (e.g., the cutting edge of a bucket attachment of the asset) based on the known geometry of the linkage of the asset and the measured joint parameters (such as angles and lengths) of the arm of the asset to which the bucket is attached. In the context of a construction asset, such as a loader or excavator, the bucket attachment is connected to the vehicle body through a series of rigid links (e.g., arms and booms) and joints (e.g., revolute or prismatic).

The linkage from the asset chassis to the bucket edge can be modeled as a serial kinematic chain. Each link and joint in the chain can be assigned a coordinate frame by the processing unit 110, and the relationship between adjacent frames is described by a transformation matrix. The revolute joints of the asset are characterized by joint angles (θ), such as the angle of rotation of the boom or arm of the asset. The prismatic joints are characterized in the model by linear displacements (d), if there are any displacements. Several parameters can be assigned by the processing unit 110 to each link of the asset (e.g., each rigid portion of the arm or attachment), including the joint angle θ (theta) (e.g., the angle of rotation about a z-axis), the link offset d (e.g., the translation distance along the z-axis), the link length a (e.g., the length of the link along the x-axis), and the link twist angle α (e.g., the angle of rotation about the x-axis). The x-axis is the axis parallel to the ground and extending in a direction parallel to the path of travel, the y-axis is the axis parallel to the ground and extending in a direction perpendicular to the path of travel, and the z-axis is the axis perpendicular to the ground.

For each joint or link, the processing unit 110 can define or utilize a homogeneous transformation matrix T_i as:

T i = [ cos ⁢ θ i - sin ⁢ θ i ⁢ cos ⁢ α i sin ⁢ θ i ⁢ sin ⁢ α i sin ⁢ θ i cos ⁢ θ i ⁢ cos ⁢ α i - cos ⁢ θ i ⁢ sin ⁢ α i 0 sin ⁢ α i cos ⁢ α i 0 0 0 ]

This transformation matrix is provided as just one example. Other transformation matrices may be created or obtained for the same or other assets. The transformation matrix can be created during calibration of the machine guidance system 100. The processing unit 110 can repeatedly examine the point cloud data to determine whether the transformation matrix is no longer accurate. For example, the reflectors may be bumped or otherwise moved from prior positions on which the transformation matrix was created. If the data points as transformed by the transformation matrix indicate a change in the position or orientation of the attachment (e.g., the bucket edge) when the attachment is not moved or was returned to a known position or orientation (that does not match the position or orientation determined from the transformation matrix), then the processing unit 110 can alert the operator using the output devices 184, 186 and/or computing device 175. The transformation matrix can then be re-defined or calibrated.

The transformation from the asset base (e.g., the asset chassis) to the bucket edge is calculated by the processing unit 110 multiplying the individual transformation matrices in order:

T base → bucket = T 1 · T 2 · T 3 ⁢ … ⁢ T n

where n represents the number of joints and/or links in the kinematic chain. The resulting matrix T_{base→bucket} represents the position of the bucket edge (e.g., the translation components in the last column of the matrix T_{base→bucket}) and the orientation of the bucket edge (e.g., the rotation submatrix that includes the upper left 3×3 submatrix in the matrix T_{base→bucket}). The processing unit 110 obtains the joint angles from the data provided by the optical sensors 120, 130 or, in some embodiments, by the movement sensor 160. The link lengths and offsets are known or measured from the geometry of the asset. These values can be input or programmed into the processing unit 110.

The processing unit 110 can use this position and orientation matrix to track the position and orientation of the attachment (or the cutting edge of the attachment) in real time. The forward kinematics model can be updated by the processing unit 110 in real time as the joints move, providing the current position and orientation of the bucket edge in the coordinate frame of the asset.

For example, the asset may be a loader with an arm (referred to as link 1) and a bucket attachment (referred to as link 2), both with revolute joints. The angle of the arm relative to the chassis of the asset can be represented by the angle θ₁, and the angle of the bucket relative to the arm can be represented by the angle θ₂. The length of the arm can be represented by a₁, and the length from the tip of the arm (to which the bucket is attached) to the cutting edge of the bucket can be represented by a₂.

The position of the bucket edge in the asset frame is then calculated by the processing unit 110 as:

x = α 1 ⁢ cos ⁢ θ 1 + α 2 ⁢ cos ⁢ ( θ 1 + θ 2 ) y = α 1 ⁢ sin ⁢ θ 1 + α 2 ⁢ sin ⁢ ( θ 1 + θ 2 )

The orientation of the bucket edge is calculated by the processing unit 110 as the rotation angle θ₁+θ₂.

At 222, the arm and attachment reflector data from 214 along with known dimensions of the asset are examined by the processing unit 110 to determine the position and orientation of the cutting edge of the bucket attachment.

At 224, the position and orientation of the asset from 218 and the position and orientation of the attachment (e.g., the cutting edge of the bucket attachment) from 222 are examined to calculate the elevation of the attachment (e.g., the cutting edge of the bucket attachment).

For example, the processing unit 110 can determine the elevation of the cutting edge of the bucket by combining the position and orientation of the loader (from 218) with the position and orientation of the cutting edge of the bucket attachment (from 222). The processing unit 110 can apply geometric transformations and kinematic relationships to map the relative position of the cutting edge to the coordinate system (e.g., the global coordinate system). The position and orientation of the asset (e.g., in X, Y, and Z coordinates) can be determined using location data and movement data. The orientation of the asset (e.g., roll, pitch, and yaw) also can be provided by the movement data and the location data. The relative position of the cutting edge of the bucket with respect to the asset can be determined using data from the optical sensors and the known relative position of the reflector on the bucket to the cutting edge of the bucket. For example, the orientation of the cutting edge (e.g., tilt angle) can be calculated or derived from the geometry of the bucket. To determine the elevation of the cutting edge, the processing unit 110 can define the position and orientation of the asset in a coordinate system (e.g., a local coordinate system). The position and orientation of the cutting edge of the bucket can be calculated relative to this local coordinate system. The processing unit 110 can use the position and orientation of the asset to transform the relative position of the cutting edge to the asset into the global coordinate system.

At 226, the elevation of the terrain can be determined from the position and orientation of the terrain from 220. The processing unit 110 can determine the terrain elevation using the position and orientation of the terrain derived in 220. The processing unit 110 examines the point cloud data generated by the optical sensors 120, 130 to calculate the elevation of the terrain at specific locations.

e. Operation without Reliance on Gnss

While the location sensors may include GNSS antennas 145, 155 and receivers 140, 150, in some embodiments, the machine guidance system 100 does not include the antennas 145, 155 or receivers 140, 150, or can operate while the location sensors are inoperable or do not have access to satellite signals. For example, the machine guidance system 100 can operate indoors or in subterranean areas without having access to GNSS (e.g., GPS) signals.

In such a situation, reflectors (e.g., passive reflectors) can be placed in known locations off-board the asset. For example, the reflectors can be placed on walls or structures as reference points for positioning. The optical sensors 120, 130 and processing unit 110 can detect these off-board reflectors using point cloud data similar to how the optical sensors 120, 130 and processing unit 110 detect the reflectors onboard the asset. In another example, the optical sensors 120, 130 and processing unit 110 can detect patterns on the reflectors or positioning can be determined using SLAM algorithms. The size and/or shape of these off-board reflectors as detected by the processing unit 110 can indicate the location of the asset to the processing unit 110. For example, if square-shaped reflectors are used, the processing unit 110 can examine the point cloud data to determine whether the reflectors appear to have a square shape, a rectangular shape, a diamond shape, or the like. These different detected shapes (as well as the detected sizes) of the off-board reflectors can indicate how far (e.g., based on detected size) and the relative location of (e.g., based on the detected shape) the asset (or the optical sensor 120 and/or 130) relative to the off-board reflector. This feature allows the machine guidance system 100 to function in environments where GNSS satellite signals are unavailable, such as underground construction sites, mines, warehouses, etc.

III. Slope and Cross-Slope Calculation Process

FIG. 3 illustrates a flowchart of one example of a method 300 for calculating the slope and/or cross-slope of terrain. With continued reference to the flowchart of the method 300 shown in FIG. 3, FIG. 4 illustrates a perspective view of a terrain surface 400, FIG. 5 illustrates an elevational view of the terrain surface 400 shown in FIG. 4, and FIG. 6 illustrates a plan view of the terrain surface shown in FIGS. 3 and 4. The method 300 can represent the operations performed by the machine guidance system 100, and specifically the processing unit 110, in calculating slope or cross-slope of a terrain surface on which the asset is performing work (e.g., grading).

At 302, reference locations 402, 404, 406, and/or 408 on the terrain surface 400 are identified or marked. These locations 402, 404, 406, and/or 408 can be identified or marked by the asset moving to a first one of the locations 402, 404, 406, and/or 408 and lowering the attachment (e.g., the cutting edge of the bucket) to the terrain surface 400 at the first location 402, 404, 406, and/or 408. In the example illustrated in FIGS. 4 through 6, the reference locations 402, 404 are used to calculate the slope, while the locations 406, 608 are used to calculate the cross-slope of the terrain surface 400. The locations 402, 404, 406, 408 can be marked by an operator actuating an input device 182 (e.g., a button, touchscreen interface, or the like) once the asset and attachment are at the locations 402, 404, 406, 408.

At 304, the horizontal locations and elevations of the reference locations 402, 404, 406, 408 are obtained. The location sensors and the optical sensors 120, 130 provide the processing unit 110 with the horizontal location (e.g., geographic location, or x- and y-coordinates) and elevation (e.g., z-coordinate) of each of the locations 402, 404, 406, and/or 408 being used. For example, the location sensors can provide the horizontal location and elevation of the antenna(s) 145, 155, and the optical sensors 120, 130 can detect the location and orientation of the cutting edge of the bucket, as described above.

At 306, a horizontal distance between the reference locations 402, 404, 406, and/or 408 used to calculate the slope or cross-slope is calculated. The processing unit 110 can calculate the distance along the x-y plane between the locations 402 and 404 for a horizontal distance 600 of the slope and between the locations 406, 408 for a horizontal distance 700 of the cross-slope.

At 308, an elevation difference between the reference locations 402, 404, 406, and/or 408 are used to calculate the slope or cross-slope is calculated. The processing unit 110 can calculate the distance in or parallel to the z-axis between the locations 402 and 404 for an elevation difference 702 of the slope and between the locations 406, 408 for an elevation difference 704 of the cross-slope.

At 310, the slope and/or cross-slope of the terrain surface 400 is calculated. The processing unit 110 can calculate the slope by dividing the elevation difference 502 by the horizontal distance 500 for the slope and can divide the elevation difference 504 by the horizontal distance 600 of the cross-slope.

In another example, the slope and/or cross-slope can be calculated using a single reference point. The single reference location 402, 404, 406, 408 can be identified or marked by the asset moving to the location and lowering the attachment (e.g., the cutting edge of the bucket) to the terrain surface 400 at the location 402, 404, 406, 408. The location and elevation of that location can be measured, as described above. The asset can then move in a direction, with the distance moved and the change in elevation measured and determined using the data provided by the GNSS receivers 140, 150 and/or movement sensor 160. The change in elevation can be divided by the change in horizontal distance to obtain the slope or cross-slope.

The slope and cross-slope are used by the processing unit 110 shape, grade, and/or finish surfaces according to defined specifications. The slope and cross-slope are used by the processing unit 110 to control how far the cutting edge of the bucket is lowered (or raised) in different geographic locations to ensure that the terrain surface 400 (as modified by the cutting edge of the bucket) is graded, leveled, or otherwise modified to the defined specification as the asset moves on the terrain surface.

The processing unit 110 uses the calculated slope and/or cross-slope to guide the operator or automatically adjust the attachment so that the terrain surface 400 being modified or constructed matches a desired slope. When cutting or filling material, the processing unit 110 uses the slope and cross-slope to maintain the correct surface to prevent over-excavation or overfilling. The processing unit 110 can repeatedly monitor and compare the slope and/or cross-slope to defined values to make sure that the finished surface matches the defined specifications for the project. If the actual position or orientation of the attachment deviates from the desired slope or cross-slope of the terrain surface 400 being modified, such as if the cutting edge of the bucket cuts too deep or not deep enough, the processing unit 110 can direct the ACU 180 to automatically stop movement of the asset, to automatically raise the attachment if the processing unit 110 determines that the attachment is digging too far into the terrain surface 400 (and at risk for contacting gas lines, electric cables, water lines, etc.), to automatically alert the operator using the output devices 184, 186 and/or the computing device 175, etc.

IV. Example Asset

FIG. 7 is a perspective view of one example of the machine guidance system 100 shown in FIG. 1 onboard an asset 500. FIG. 8 is a top plan view of the machine guidance system 100 and the asset 500 shown in FIG. 7. The asset 500 is illustrated as a track loader that may be manually, semi-autonomously, and/or autonomously operated using the machine guidance system 100 described above. It should be understood that this example embodiment is provided to describe the various capabilities of the machine guidance system 100 and not to limit all embodiments of the inventive subject matter described herein.

The asset 500 includes a main body 510 connected to a lift arm 512, which is in turn connected to a bucket attachment 514. Other types of attachments may be attached to the lift arm 512, such as a tooth bucket, a mower, a dozer blade, a soil conditioner, a grapple, a trencher, or the like. The asset 500 also includes a cab 516 that provides an enclosure from which the operator can operate the asset 500. The asset 500 further includes a track 518 that enables movement of the asset 500 across rugged terrain. In some embodiments, the asset 500 may include wheels to move.

With continued reference to the asset 500 and the machine guidance system 100 shown in FIGS. 7 and 8, FIG. 9 illustrates a perspective view of a machine guidance assembly 520. The machine guidance system 100 of the asset 500 includes a machine guidance assembly 520 rigidly mounted on top of the cab 516. In some embodiments, the machine guidance assembly 520 may be mounted or located elsewhere on or in the asset 500. The machine guidance assembly 520 includes a rigid plate 522 on which is mounted a ruggedized enclosure 524 that provides isolated interfaces to the processing unit 110 and movement sensor 160, the communication network device 165, the front GNSS receiver 140, the rear GNSS receiver 150, and a power splitter 190. The cover of the ruggedized enclosure 524 is removed in FIG. 9 to show these components.

Also mounted to the rigid plate 522 is the front GNSS antenna 145 and the front optical sensor 120. In this example, the front GNSS antenna 145 is mounted on the rigid plate 522 at a location that is as far forward as possible along the x-axis of the asset 500 and generally centered along the y-axis of the asset 500. The front optical sensor 120 can be mounted on the rigid plate 522 at a location that allows the door of the cab 516, if so configured with a door, to be opened and closed, avoids contact with the arm 512 as the arm 512 is raised and lowered, and provides a line of sight to the arm/attachment joint when the arm 512 is lowered.

As shown in FIGS. 7 and 8, the asset 500 includes the rear GNSS antenna 155 mounted at the rear of the main body 510. The rear antenna 155 can be centrally located at the rear of the main body 510 to be oriented in a straight-line with the front antenna 145 along the x-axis of the asset 500. In some embodiments, the locations of the front and rear antennas 145, 155 could both be shifted along the y-axis of the asset 500 so long as the straight-line orientation between the antennas 145, 155 is maintained. In some embodiments, the location of the rear antenna 155 could be offset with respect to the front antenna 145 along the y-axis of the asset 500 as long as the location of the rear antenna 155 can be calibrated based on data from the optical sensor 120 and/or the optical sensor 130.

The machine guidance system 100 onboard the asset 500 also can include the rear optical sensor 130 mounted at or toward the rear of the main body 510. The rear optical sensor 130 can be located on the left side of the rear of the main body 510 to provide a line of sight to a first attachment reflector 544 placed on the left side of the bucket attachment 514 and a second arm reflector 546 placed on the left side of the lift arm 512 of the asset 500. The reflectors 544, 546 may be onboard the asset 500 in that the reflectors 544, 546 are mounted to the asset 500 or the attachment 514 that is coupled with the asset 500. The onboard reflectors 544, 546 may be passive reflectors as described above. The reflectors 544, 546 can be positioned to be within the field of view of the front optical sensor 120 such that one or more than one line of sight exists between the front optical sensor 120 and each of the reflectors 544, 546, or at least one of the reflectors 544, 546. In some embodiments, the front and rear optical sensors 120, 130 track movement and/or positions of the lift arm 512 and bucket attachment 514 throughout their entire range of movement, and can be positioned to provide a 360-degree field of view around the asset 500.

The asset 500 further includes the 5G/LTE/WiFi antenna 170 mounted at a fixed location on top of the cab 516. The asset 500 includes a first harness 550 that connects the machine guidance assembly 520 to the power adapter 195 and computing device located within the cab 516, as well as a second harness 552 that connects the machine guidance assembly 520 to the rear antenna 155 and rear optical sensor 130. In some embodiments, all of the components of the machine guidance assembly 520, the front antenna 145, the front optical sensor 120, the rear antenna 155, and the rear optical sensor 130 are rigidly attached to the asset 500 so that the deflections are less than 0.1 millimeters with 5G shock and vibration. Also, the asset 500 also includes the ACU 180 and various components located within the cab 516, including the onboard computing device 175, the input device 182, output devices 184, 186, and the power adapter 195.

One or more examples of the methods and processes described herein include a method that can include positioning an attachment of a construction vehicle at a first reference point on a terrain, obtaining, by a location sensor, first location data indicative of a first geographic position and first elevation of the first reference point, monitoring movement of the construction vehicle away from the first reference point while measuring a horizontal distance moved and a change in elevation from the first elevation, calculating, by a processing unit, a slope parameter based on the change in elevation and the horizontal distance moved from the first reference point, generating, by the processing unit, guidance information indicative of adjustments required to align the attachment with the slope parameter that is calculated, and outputting the guidance information to an operator interface for real-time feedback on adjustment of positions of the attachment or to a control system for automatic adjustment of the attachment.

The method also can include positioning the attachment at a second reference point on the terrain, and obtaining, by the location sensor, second location data indicative of a second geographic position and second elevation of the second reference point. The change in elevation can be an elevation difference between the first reference point and the second reference point. The horizontal distance can be calculated as the horizontal distance between the first reference point and the second reference point. The elevation difference of the slope parameter can be a first elevation difference, and the method also can include calculating, by the processing unit, a cross-slope parameter based on a second elevation difference between the first reference point and the second reference point along a direction that is transverse to the horizontal distance. The guidance information generated by the processing unit also can indicate the adjustments required to align the attachment with the cross-slope parameter that is calculated. The slope parameter can be calculated by the processing unit as a percentage grade based on the elevation difference divided by the horizontal distance between the first reference point and the second reference point. The method also can include automatically adjusting the positions and orientation of the attachment using actuators to align the attachment with the slope parameter that is calculated. The first reference point and the second reference point can be marked by the operator using physical buttons or a touchscreen interface onboard the construction vehicle.

The method also can include detecting errors in slope alignment of the attachment during operation of the construction vehicle based on deviations from the slope parameter and providing corrective guidance to the operator.

One or more examples of the machine guidance systems described herein include a machine guidance system that can include a location sensor configured to obtain geographic position and elevation data for at least two reference points on a terrain while onboard the construction vehicle, and a processing unit electrically coupled to the location sensor and configured to calculate a slope parameter based on an elevation difference and horizontal distance between the at least two reference points. The processing unit can generate guidance information indicative of adjustments required to align an attachment of the construction vehicle with the slope parameter that is calculated. The processing unit can output the guidance information to an operator interface or a control system.

The machine guidance system also can include the operator interface that can display the guidance information to an operator in real time for guiding operation of the attachment of the construction vehicle. The machine guidance system can include the control system that can automatically adjust the position and orientation of the attachment based on the guidance information.

The processing unit can calculate a cross-slope parameter based on another elevation difference perpendicular to the horizontal distance. The processing unit can generate the guidance information with the slope parameter and the cross-slope parameter that are calculated. The processing unit can calculate the cross-slope parameter by analyzing elevation differences across multiple points perpendicular to the horizontal distance.

The processing unit can calculate the slope parameter is calculated as a percentage grade based on the elevation difference divided by the horizontal distance between the at least two reference points. The location sensor can mark at least two reference points responsive to an operator using physical buttons or a touchscreen interface onboard the construction vehicle. The processing unit can detect errors in slope alignment of the attachment during operation of the construction vehicle based on deviations from the slope parameter and provide corrective guidance to the operator.

One or more examples of the processes and methods described herein include a method that can include positioning a bucket of a construction vehicle at a first reference point on a terrain, obtaining, by a location sensor, first location data indicative of a first geographic position and first elevation of the first reference point, positioning the bucket at a second reference point on the terrain, obtaining, by the location sensor, second location data indicative of a second geographic position and second elevation of the second reference point, calculating, by a processing unit, a slope parameter along a slope direction based on a first elevation difference and a horizontal distance between the first reference point and the second reference point, calculating, by the processing unit, a cross-slope parameter based on a second elevation difference perpendicular to the slope direction, generating, by the processing unit, guidance information indicative of adjustments required to align the bucket with the slope parameter and the cross-slope parameter that are calculated, and outputting the guidance information to an operator interface for real-time feedback on adjustment of positions of the bucket or to a control system for automatic adjustment of the bucket.

The cross-slope parameter can be calculated by analyzing multiple ones of the second elevation difference across multiple points perpendicular to the slope direction. The slope parameter can be calculated by the processing unit as a percentage grade based on the first elevation difference divided by the horizontal distance between the first reference point and the second reference point.

The method also can include automatically adjusting the positions and orientation of the bucket using actuators to align the bucket with the slope parameter and the cross-slope parameter that are calculated. The method can include detecting errors in slope alignment of the bucket during operation of the construction vehicle based on deviations from one or both the slope parameter or the cross-slope parameter, and providing corrective guidance to the operator.

References to “one embodiment,” “an embodiment,” “an example embodiment,” or “embodiments” mean that the feature or features being described are included in at least one embodiment of a machine guidance system deployed on a construction vehicle. Separate references to “one embodiment,” an embodiment, “an example embodiment,” or “embodiments” in this disclosure do not necessarily refer to the same embodiment and are also not mutually exclusive unless so stated and/or except as will be readily apparent to one of ordinary skill in the art from the disclosure. For example, a feature, structure, function, etc. described in one embodiment may also be included in other embodiments, but is not necessarily included. Thus, a machine guidance system or method can include a variety of combinations and/or integrations of the features, structures, functions, etc. described herein.

The embodiments disclosed herein are intended to encompass various modifications, substitutions, and rearrangements that would be apparent to those of ordinary skill in the art, and such variations are considered to fall within the scope of the disclosed subject matter. For example, the described systems may be adapted for use with different types of construction vehicles, attachments, or environmental conditions, and the methods may be implemented using alternative hardware or software configurations without departing from the spirit of the disclosed subject matter.

In this disclosure, the use of any and all examples or exemplary language (such as “for example”) is intended merely to better describe the embodiments and does not pose a limitation on the scope of all embodiments of the inventive subject matter. No language in the disclosure should be construed as indicating any non-claimed element essential to the practice of the inventive subject matter.

Also, the use of the terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a system, device, or method that comprises a list of elements does not include only those elements, but may include other elements not expressly listed or inherent to such system, device, or method.

Further, the use of relative relational terms, such as first and second, are used solely to distinguish one unit or action from another unit or action without necessarily requiring or implying any actual such relationship or order between such units or actions.

Finally, while the inventive subject matter has been described and illustrated hereinabove with reference to various example embodiments, it should be understood that various modifications could be made to these embodiments without departing from the scope of the invention. Therefore, the inventive subject matter is not to be limited to the specific structural configurations or methodologies of the example embodiments, except insofar as such limitations are included in the following claims.