Patent application title:

METHOD FOR VISUALIZING THE ENVIRONMENT OF A UTILITY VEHICLE AND UTILITY VEHICLE HAVING AN ENVIRONMENTAL VISUALIZATION SYSTEM

Publication number:

US20250290297A1

Publication date:
Application number:

18/603,418

Filed date:

2024-03-13

Smart Summary: A system helps utility vehicles, like those with front loaders, see their surroundings better. It uses two cameras: one is placed in front of the vehicle, and the other is on the cab. These cameras capture images of the area around the vehicle. A data processing unit then combines and processes these images. Finally, the processed images are shown on a display screen for the driver to see. 🚀 TL;DR

Abstract:

A method for visualizing an environment of a utility vehicle equipped with a front loader via a visualization system, includes generating first image data using a first camera mounted on a front region of the utility vehicle in front of the cab of the vehicle, generating second image data using a second camera mounted on the cab, processing the first and second image data using a data processing unit, and optically displaying the processed first and second image data using a display unit.

Inventors:

Applicant:

Interested in similar patents?

Get notified when new applications in this technology area are published.

Classification:

E02F9/262 »  CPC main

Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups  - ; Indicating devices; Surveying the work-site to be treated with follow-up actions to control the work tool, e.g. controller

E02F3/431 »  CPC further

Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms , e.g. dippers, buckets; Component parts; Drives for dippers, buckets, dipper-arms or bucket-arms; Control of dipper or bucket position; Control of sequence of drive operations for bucket-arms, front-end loaders, dumpers or the like

E02F9/26 IPC

Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups  -  Indicating devices

E02F3/43 IPC

Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms , e.g. dippers, buckets; Component parts; Drives for dippers, buckets, dipper-arms or bucket-arms Control of dipper or bucket position; Control of sequence of drive operations

Description

FIELD OF THE DISCLOSURE

The disclosure relates to a method for visualizing an environment of a utility vehicle. Furthermore, the disclosure relates to a utility vehicle having a system for visualizing an environment of the utility vehicle.

BACKGROUND

In agricultural utility vehicles, it is known to use a camera mounted on the utility vehicle to visualize its surroundings and to process the image data of the camera. The processed image data is then used to visualize the vehicle environment.

SUMMARY

The object of the present disclosure is to provide a technically simple environment visualization for utility vehicles, which efficiently supports the driving of the utility vehicle.

This object is achieved by a method having the features of one or more of the following embodiments.

Further advantageous refinements of the method according to the disclosure can be found in one or more of the following embodiments.

According to an embodiment, an environment of a utility vehicle equipped with a front loader is visualized. For this purpose, a visualization system is provided, which comprises a first camera, a second camera, a data processing unit, and a display unit. The first camera is mounted on a front region of the utility vehicle (directly or indirectly) in front of its cab. The second camera is mounted (directly or indirectly) on the cab.

By means of the first and second camera, image data can be generated, which is processed in the data processing unit. This data processing unit is connected for data transfer to a display unit, which visualizes or optically displays the processed image data.

Depending on the technical design, the data processing unit may be partially or completely integrated in the display unit or may be designed as a completely physically separate unit.

The display unit can be used to visually display or visualize an environment of the utility vehicle. The two differently positioned cameras can be used to capture different regions of the environment.

With the first camera, relatively low environmental regions can be captured well, for example a loading tool of the front loader, its load content and the front edge of the tool, a region near the ground and low-level working points of the front loader. The front edge of the engine hood or of the cow catcher, the coupling points of the front hitch on the vehicle and on the accessory equipment, the environment region in front of the utility vehicle, in front of its engine hood and in the region of its front tires can also be easily captured and visualized.

Using the second camera, relatively high regions of the environment can be captured well, for example, high working points of the front loader.

The cameras can be used to avoid potential blind spot regions and regions that are poorly visible in the field of view of the utility vehicle driver or user. The visibility of the environment from the cab, and for example the working area of the front loader as well as a loading tool mounted on it (e.g. a bucket, bite bucket, fork-lift, pallet fork-lift, clamping pliers, pitchfork), are significantly improved for a machine operator. For example, particularly low and particularly high working areas of the front loader have improved visibility.

Turning and maneuvering while driving is also facilitated with the visualization of the surroundings. In addition, the visualization allows the area in front of the engine hood and the front loader to also be viewed for example, even at low levels. This allows the machine operator to attach the respective loading tool to the front loader from the cab very precisely and in a short time. Similarly, when mounting attachments onto the front hitch, the operator is assisted by the improved visibility of the hitch coupling points from the cab.

Overall, the visualization of the environment in the surrounding area in front of the utility vehicle assists a machine operator in a technically simple way in carrying out various activities (e.g. driving, loading, mounting a loading tool), so that the activities can be carried out in a time-saving and precise manner and any damage to the utility vehicle, the loading tool, personnel or the environment are particularly simple to avoid.

The visualization system is for example designed such that the processed image data of only the first camera or of only the second camera or of both cameras simultaneously are selectively displayed on the display unit. This enables the person operating the machine to select the relevant environment to be displayed or visualized according to each specific job. This supports efficient operation of the utility vehicle during different work processes. For the selection (e.g. manually or voice-controlled), the visualization system for example has a suitable selection function, for example in the form of a switching unit upstream of the display unit. On the other hand, automation of the selection function that further relieves the burden on the machine operator is conceivable, by an automatic changeover being carried out, for example depending on the determined height of the loading tool of the front loader.

For example, an actual value of a physical quantity of the front loader is determined by means of the data processing unit and displayed optically on the display unit. This provides the machine operator with additional information that facilitates the various tasks with the utility vehicle. The physical quantity is, for example, an inclination angle or a height of the loading tool. The actual value of the inclination angle can then be displayed as a line inclined at the same angle as the current inclination angle. The actual value of the height can be displayed in a bar chart as a horizontal line between a minimum height (0%) and a maximum height (100%).

The actual value of the physical quantity can be determined in a technically simple way if at least one optical marker on the front loader or on its loading tool and/or known kinematics data of the front loader or its loading tool is taken into account. At least one optical marker is arranged on the front loader. This at least one optical marker can be captured or detected by one camera or both cameras of the visualization system. From the detected position of the optical marker(s) and known kinematics data of the front loader, the data processing unit can then determine the actual value of the physical quantity. Apart from such an optically based approach, the actual value of the physical quantity can be additionally or alternatively determined by conventional means using the position and/or acceleration sensors assigned to the front loader.

The driver receives additional assistance while working with the front loader by the visual display of a predetermined setpoint value of the physical quantity of the front loader, e.g. a height of the front loader or of its loading tool or an inclination angle of the loading tool, on the display unit. If the actual value of the physical quantity is also displayed, the machine operator can control the front loader very precisely toward the setpoint value in a very short time.

The aforementioned setpoint value of the physical quantity is entered in a suitable manner into a control system of the utility vehicle, for example via a user interface for data entry or by touching the screen of the display unit itself.

The movement of the front loader toward the setpoint value is controlled, for example, by means of an actuating device provided as standard in the cab, (e.g. a joystick), which is operated by the machine operator.

In an embodiment, the visualization system is connected via a data and/or control bus (e.g. CAN) of the utility vehicle to a control unit (e.g., a controller including a processor and memory) for controlling the movement of the front loader, for example its loading tool. The control unit controls, for example, suitable actuators (e.g. hydraulic lifting cylinders) of the front loader. The data and/or control bus then also serves as a data interface between the control unit and the display unit of the visualization system. The setpoint values that can be identified on the display unit are then also available to the control unit. The visualization system uses this feature to perform an automatic transition of the front loader or its loading tool to a predetermined setpoint position (e.g. height, angle). This automated motion control of the front loader can be simply triggered, for example, by actuating a signaling device (e.g. button, contact panel, push button) on a user interface in the cab. This relieves the machine operator of manual movement control of the front loader.

In various loading operations (e.g. with a trailer or feed mixer) it is advantageous if the machine operator receives additional auxiliary information in order to better assess a relationship (e.g. height, angle, distance) of the front loader or its loading tool to an object in the vicinity of the utility vehicle (e.g. trailer, feed mixer, loaded goods pile). This is particularly relevant at very high and very low positions of the front loader and its loading tool, to achieve a precise interaction with the object with minimum maneuvering effort and to avoid unwanted collisions between the loading tool and such an object.

Therefore, the visualization system or its camera(s) is for example used to detect an object in the vicinity of the utility vehicle. Depending on the detected object, the data processing unit is used to derive auxiliary information which is displayed on the display unit.

For example, an expected future direction of movement of the loading tool in the direction of the object is determined by means of the data processing unit. The auxiliary information can then be derived from this direction of movement and displayed on the display unit.

The auxiliary information for example includes at least one trajectory line, which represents an expected future trajectory of the loading tool and is displayed on the display unit. To provide visually convenient support for the machine operator, the at least one trajectory line is displayed on the display unit, for example, in such a way that it is parallel to the ground, begins at a lower edge or front edge of the loading tool and runs in the direction of the object. A precise mathematical determination and visual representation of the trajectory line relative to the bottom or front edge of the loading tool is possible if the optical marker and known kinematics data of the front loader, as described above, are taken into account, for example for calculating the height and/or angle of the loading tool.

Further for example, the auxiliary information includes a collision line, which represents an expected collision region or collision position between the loading tool and the object and is visualized on the display unit. This allows the machine operator to be visually and/or acoustically warned of a potential collision between the front loader or loading tool and the object in good time. Advantageously, to obtain a mathematical derivation of the collision line the expected future trajectory of the loading tool is compared with depth information (e.g. loading height, distance from the object), the depth information for example being obtained from stereo image data using both cameras. To obtain the depth information, at least one of the two cameras can be designed as a stereo camera. If the cameras are mono cameras, they can also perform the function of a stereo camera by appropriately combining the generated image data.

The object of the disclosure is also achieved by a utility vehicle having the features of one or more of the following embodiments.

Further advantageous embodiments of the utility vehicle according to the disclosure can be found in one or more of the following embodiments.

According to an embodiment, the utility vehicle has a front loader. In addition, the utility vehicle has a system for visualizing the environment of the utility vehicle. To generate image data, this visualization system contains a first camera which is mounted on a front region of the utility vehicle in front of its cab. Furthermore, for generating image data the visualization system contains a second camera which is mounted on the cab. In addition, the visualization system contains a data processing unit for processing the image data and a display unit for visualizing or optically displaying the processed image data.

As already explained, the visualization system can be used to avoid blind spot regions and regions of poor visibility in the field of view of the machine operator. The environment visualization in the front region of the utility vehicle supports a machine operator in a technically simple way in performing various activities (e.g. driving, loading, mounting a loading tool). The activities can therefore be carried out in a time-saving and accurate manner. Any damage to the utility vehicle, the loading tool, persons, or the environment is particularly easily avoided.

The utility vehicle is for example an agricultural utility vehicle, such as a tractor. The front loader for example has a loading tool, which is designed, for example, as a bucket, bite bucket, fork-lift, pallet fork-lift, clamping pliers or pitchfork.

The first camera does not necessarily have to be mounted directly on the front region (e.g. the support structure) of the utility vehicle in front of its cab. Instead, in some embodiments the first camera can also be mounted on the front region indirectly via a suitable holding device.

Similarly, the second camera is also not necessarily mounted directly on the cab of the utility vehicle. Instead, in some embodiments the second camera can also be mounted indirectly on the vehicle cab via a suitable holding device.

For example, the front region for the direct or indirect mounting of the first camera is the engine hood of the utility vehicle. The second camera is advantageously mounted directly or indirectly on the roof of the vehicle cab. Such positions of the cameras support the visualization of those regions of the utility vehicle environment that are particularly important to the driver or the machine operator during the various tasks.

For example, the first camera has an opening angle of approximately 120°. As a result, the machine operator receives a generous visualization of the environment in the area around the utility vehicle near to the ground without being distracted by a too wide-ranging environment visualization.

The second camera has an opening angle of about 90°, which gives the machine operator a generous visualization of the environment in the region of high working points of the front loader, without being distracted in this region by a too wide-ranging environment visualization.

For example, the first camera is mounted on the front region of the utility vehicle by means of a holding device, i.e. indirectly. The holding device is for example rod-shaped and therefore designed to be very compact, so that the holding device (and thus the first camera) can also be mounted at different points of the utility vehicle, accessory equipment, or trailer if corresponding predetermined mounting points for the releasable mounting of the holding device are available there. In another embodiment, the same predetermined mounting point is also suitable for a releasable mounting of one of the cameras. Alternatively, at least one further predetermined mounting point may be provided on the utility vehicle, which is only suitable for a releasable mounting of a camera and not for the holding device. These different mounting options allow the user to position the cameras where they are needed by the user for the most efficient environment visualization.

A predetermined fixing point is arranged, for example, on the cow catcher or on the engine hood.

The holding device may be designed for receiving different camera types and/or for a pivotable mounting of the camera, so that the camera opening angle can be aligned in an application-specific way. Also, in an embodiment the holding device itself can be arranged on the front region of the utility vehicle in different inclined positions.

The holding device is for example designed to be length-adjustable. This provides very flexible support for individual, application-specific positions of the first camera.

In an embodiment, the holding device can be bent at an angle so that the holding device is mechanically stable and robust against possible collisions with obstacles. Any damage to or destruction of the holding device and/or the first camera can thus be avoided with little technical effort.

For example, the bending is designed to be reversible, whereby after the holding device bends, the mounted camera can be tracked back to the original position in a user-friendly manner.

For example, the holding device is composed along its longitudinal direction of a plurality of longitudinal sections, which are held together in the longitudinal direction by a tensioning or spring force. In case of contact with an obstacle and critical impact force, the resistance of the tensioning or spring force is overcome and only the part of the holding device with the contact with the obstacle is bent. After force is no longer being applied to the bent part of the holding device, it automatically straightens itself due to the tensioning or spring force and thus causes the reversible bending.

In another embodiment of the reversible bending, the holding device has a folding mechanism of such a kind that after contact with an obstacle and a corresponding impact force, the holding device automatically folds away, e.g. in the direction of the engine hood of the utility vehicle. After this collision, the holding device can then be realigned manually or electrically, for example.

For example, the display unit of the visualization system is permanently installed in the utility vehicle. Alternatively, a mobile, portable display unit (e.g. tablet, smartphone) can be used. The data processing unit may be designed as a physically completely separate unit. Alternatively, in some embodiments the data processing unit is partially or completely integrated in the display unit, so that the connection to the cameras is technically simple.

In another alternative embodiment, the display unit has an external screen, which is connected to a vehicle-internal or external computing unit or data processing unit.

For example, the user can select whether the data streams of both cameras or of only one of the two cameras is/are to be visualized on the display unit. For example, the display unit has a magnification function (e.g. by manual touch on the visual display surface of the display unit), with which selected areas of the visualized image can be magnified.

The components of the visualization system can be partially or completely connected to one another via data cables. For their electronic operation, the components are advantageously connected to an electrical power source of the utility vehicle, where necessary. Alternatively, the aforementioned components can be partially or completely wirelessly interconnected (e.g. WiFi, WLAN). In this case, the cameras and optionally other components of the visualization system are for example equipped with battery operation.

The above and other features will become apparent from the following detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The method according to the disclosure and the utility vehicle according to the disclosure will be explained in more detail below with reference to the accompanying drawings. Component parts of equivalent or comparable function are identified by the same reference signs. In the drawings:

FIG. 1 shows an example embodiment of the utility vehicle according to the disclosure in a side view;

FIG. 2 shows a side view of the front region of the utility vehicle according to the disclosure in a further example embodiment;

FIG. 3 shows a representation in the form of a block circuit diagram of a system for visualizing an environment of the utility vehicle according to the disclosure;

FIG. 4 shows an optical display of the image data of the vehicle environment generated by the visualization system;

FIG. 5 shows an optical display of the image data of the vehicle environment generated by the visualization system and derived auxiliary information; and

FIG. 6 shows an optical display according to FIG. 5 and of further auxiliary information.

DETAILED DESCRIPTION

The embodiments or implementations disclosed in the above drawings and the following detailed description are not intended to be exhaustive or to limit the present disclosure to these embodiments or implementations.

FIG. 1 shows a utility vehicle 10 in the form of a tractor with a front loader 12. The loading tool 14 thereof is designed in FIG. 1 as a bucket 16. The utility vehicle 10 carries two schematically illustrated cameras 18, 20. The two cameras 18, 20 are components of a visualization system 22 for visualizing an environment of the utility vehicle 10, explained in more detail with reference to FIG. 3. This environment visualization supports the machine operator or the driver of the utility vehicle 10 in carrying out various activities (e.g. loading, transporting, driving) with the utility vehicle 10.

The first camera 18 is mounted in a manner not shown in detail in FIG. 1 on a front region 24 of the utility vehicle 10 in front of its driver's cab 26. The mounting of the first camera 18 can be implemented on the utility vehicle 10 directly or indirectly via a holding device 28 (FIG. 2). For example, the first camera 18 is mounted on the engine hood 30 of the utility vehicle 10, e.g. indirectly via a suitable holding device not shown in FIG. 1 or indirectly via the holding device 28 according to FIG. 2 in the front region of the engine hood 30. In this front region, a predetermined mounting point 32 is provided for a releasable mounting of the holding device 28. On the utility vehicle 10, further predetermined fixing points 32 may also be arranged to allow individual positioning of the holding device 28. Also, a mounting point 32 may be designed such that it is suitable for a direct releasable mounting of one of the two cameras 18, 20 without a holding device 28.

The second camera 20 is secured to the front region of the cab roof 34 of the cab 26 by means of a suitable mounting point 32. The second camera 20 is thus releasably mounted directly on the utility vehicle 10 without a holding device 28. The mounting point 32 for the second camera 20 is only shown schematically in FIG. 2.

The cameras 18, 20 are for example pivotably mounted at their respective mounting points 32 or on the holding device 28, so that they can be optimally aligned with the utility vehicle 10 for each specific task.

The first camera 18 in FIG. 1 has an opening angle W1 greater than 90°. In another embodiment, the opening angle W1 is approximately 120°. The opening angle W2 of the second camera 20 in FIG. 1 is less than 90°. In another embodiment, the opening angle W2 is approximately 90°.

In FIG. 1, a field of view 36 of the machine operator from the driver's cab 26 is shown by an opening angle in dashed lines. Between this field of view 36 and the ground or road surface 38 of the utility vehicle 10 are blind spot regions that cannot be viewed by the machine operator personally. On the other hand, these blind spot regions can be visualized by means of the first camera 18. This significantly relieves the burden on the machine operator while performing various tasks with relatively low working points 40 in front of the front region 24 and below the field of view 36.

Similarly, the machine operator is significantly relieved during various tasks with relatively high working points 42 above the field of view 36 by having the image data of the second camera 20 visualized for the machine operator.

According to FIG. 2, the holding device 28 is rod-shaped, with a length that is for example adjustable (e.g. telescopic). Furthermore, the holding device 28 is designed such that it can be bent by means of a folding mechanism, not shown in detail. When the camera 18 is activated, the holding device 28 is usually located in a working position P1. In the event of contact of the holding device 28 with an obstacle, the holding device 28 bends rearward under a certain impact force in the direction of the engine hood 30 into a bent position P2, which is indicated by a dashed-line representation of the holding device 28. Unwanted damage to the holding device 28 and/or the first camera 18 is thus prevented in a technically simple manner. For example, manually or electrically, the holding device 28 can be returned to its working position P1, which means the bending takes place reversibly.

The visualization system 22 according to FIG. 3 comprises, inter alia, the two cameras 18, 20, a data processing unit 44 and a display unit 46. The data processing unit 44 and the display unit 46 here are integrated in a single physical unit, which is for example permanently installed in the utility vehicle. In other embodiments, these two functions can also be provided in physically separate units. An example of this is an optional vehicle-internal or external computing unit 48 and an external screen 50 connected thereto.

The two cameras 18, 20 generate image data D1, D2 of the environment of the utility vehicle 10. These sets of image data D1, D2 are processed in the data processing unit 44 and then optically displayed or visualized in the display unit 46 for the machine operator.

By means of an intermediate switching unit 52, the machine operator or the user can select whether only the image data D1 of the first camera 18, only the image data D2 of the second camera 20 or the image data D1, D2 of both cameras 18, 20 is to be visualized on the display unit 46.

Alternatively, the switching unit 52 implements a further automation of the selection function for supporting the machine operator, for example by automatic switching depending on the calculated height of the loading tool 14.

The components 18, 20, 44, 46, 52 mentioned are connected to one another in FIG. 3 via data cables 54. The optional components 48, 50—instead of the components 44, 46—would then be connected to the other components by a data cable 54.

Where necessary for their technical operation, the components of the visualization system 22 are connected to an electrical power source 56. If the data processing unit 44 and the display unit 46 have a different or internal power supply, they do not need to be connected to the power source 56. Alternatively, these components 44, 46 of the visualization system 22 can also be connected to the power source 56.

In a further embodiment, the power source 56 is redundant, at least for individual components of the visualization system 22, if these components (e.g. the cameras 18, 20) are wirelessly connected to the switching unit 52 and/or to the data processing unit 44 for data transfer and contain an internal electrical power source (e.g. battery operation).

In FIG. 4, the display unit 46 visualizes the environment of the utility vehicle 10 captured with the first camera 18. Two physical quantities of the front loader 12 are also visualized there, namely an inclination angle W_L of the bucket 16 (e.g. relative to a horizontal) and a height H of the bucket 16 (e.g. relative to the ground or driving surface 38). The physical quantities W_L, H and their actual values W_ist, H_ist determined by the data processing unit 44 are visualized by means of suitable (bar) charts 58. The actual value W_ist of the inclination angle W_L is 5% and is represented by a correspondingly inclined bar. The actual value H_ist of the height H is represented by a horizontal bar and lies between the minimum height (0%) and the maximum height (100%).

The respective actual value of the physical quantity W_L, H is advantageously determined in the data processing unit 44 as a function of at least one detected optical marker M and known kinematics data D_k of the front loader 12. The optical marker(s) M is/are located on the front loader 12. In FIG. 4, two markers M arranged on the front loader 12 are detected by the first camera 18.

In addition, a setpoint value W_soll, H_soll of the quantities W_L, H respectively can also be displayed on the diagrams 58. For example, the setpoint value is represented by a colored triangle or a colored arrowhead. The respective setpoint value can be entered, for example, via a user interface and processed by the data processing unit 44. However, the setpoint values can also be available as previously stored values and be received by the data processing unit 44. Alternatively, the respective setpoint value can be entered by touching the bar chart 58 and visualized there. With the visualized setpoint value W_soll or H_soll, a machine operator is supported in changing the actual value in the direction of the setpoint value in a user-friendly manner by means of an actuating device (e.g. joystick).

In a further embodiment, a standardly available data and/or control bus 60 (e.g. CAN) of the utility vehicle 10 acts as a kind of interface between the visualization system 22 and a control unit 62 (e.g., a controller including a processor and memory) for activating the front loader 12 or its loading tool 14. In this case, the visualization system 22 is for example connected to the bus 60 via the data processing unit 44. The setpoint values W_soll, H_soll, which can be identified on the display unit 46, are then also available to the control unit 62. This enables an automated transition of the front loader 12 or its loading tool 14 into a predetermined setpoint position (e.g. height, angle), for example by activating a signaling means (e.g. button, contact panel, push button) on a user interface in the cab 26. This relieves the machine operator of manual movement control of the front loader 12.

In FIGS. 5 and 6, additional auxiliary information is visualized on the display unit 46, which enables the machine operator to carry out the various (loading) tasks with the front loader 12 even more efficiently.

In this case, by means of the visualization system 22 or its cameras 18, 20—here by means of the first camera 18—an object 64 surrounding the utility vehicle 10 and its front loader 12 is detected. The object 64 here is a pile of gravel or rubble. The auxiliary information yet to be explained is helpful, for example, for the intended task of loading the gravel or rubble pile 64.

In this case, the auxiliary information is derived from the data processing unit 44 depending on the detected object 64 and visualized on the display unit 46.

For example, the auxiliary information is derived by firstly determining an expected future direction of movement of the loading tool 14, 16 in the direction of the object 64 by means of the data processing unit 44. The auxiliary information is then derived depending on this direction of movement.

Thus, for example, two trajectory lines 66 are calculated as auxiliary information and visualized on the display unit 46. The two trajectory lines 66 represent an expected future trajectory of the loading tool 14, 16 in the direction of the object 64. The trajectory lines 66 are for example calculated as lines running approximately parallel to the ground surface 40. They begin at a lower edge 68 of the loading tool 14 or the bucket 16 and are aligned to run in the direction of the object 64.

As further auxiliary information, certain distances 70 are visualized on the display unit 46, which correspond to a distance (e.g. 1 meter, 2 meters, 3 meters) between the bucket lower edge 68 and a position on the object 64.

In FIG. 6, further auxiliary information in the form of a collision line 72 is visualized on the display unit 46, which represents an expected collision region between the loading tool 14, 16 or its lower edge 68 and the object 64.

In FIGS. 4 to 6, the illustrated environment visualizations are produced for example by using the image data D1 of the first camera 18. Analogously, using the image data D2 of the second camera 20, an environment visualization can be generated for relatively high working points 42, for example above the field of view 36.

The environment visualization explained above significantly relieves the burden on the machine operator during various tasks and thus enables the tasks to be carried out with the utility vehicle 10 more comfortably and significantly more efficiently.

For the sake of good order, it is to be noted that all illustrated details and components are not necessarily disclosed to scale. Individual details are sometimes only shown schematically and/or not to scale.

The terminology used herein is for the purpose of describing example embodiments or implementations and is not intended to be limiting of the disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the any use of the terms “has,” “includes,” “comprises,” or the like, in this specification, identifies the presence of stated features, integers, steps, operations, elements, and/or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Those having ordinary skill in the art will recognize that terms such as “above,” “below,” “upward,” “downward,” “top,” “bottom,” etc., are used descriptively for the drawings, and do not represent limitations on the scope of the present disclosure, as defined by the appended claims. Furthermore, the teachings may be described herein in terms of functional and/or logical block components or various processing steps, which may include any number of hardware, software, and/or firmware components configured to perform the specified functions.

Terms of degree, such as “generally,” “substantially,” or “approximately” are understood by those having ordinary skill in the art to refer to reasonable ranges outside of a given value or orientation, for example, general tolerances or positional relationships associated with manufacturing, assembly, and use of the described embodiments or implementations.

As used herein, “e.g.,” is utilized to non-exhaustively list examples and carries the same meaning as alternative illustrative phrases such as “including,” “including, but not limited to,” and “including without limitation.” Unless otherwise limited or modified, lists with elements that are separated by conjunctive terms (e.g., “and”) and that are also preceded by the phrase “one or more of” or “at least one of” indicate configurations or arrangements that potentially include individual elements of the list, or any combination thereof. For example, “at least one of A, B, and C” or “one or more of A, B, and C” indicates the possibilities of only A, only B, only C, or any combination of two or more of A, B, and C (e.g., A and B; B and C; A and C; or A, B, and C).

While the above describes example embodiments or implementations of the present disclosure, these descriptions should not be viewed in a restrictive or limiting sense. Rather, there are several variations and modifications which may be made without departing from the scope of the appended claims.

Claims

What is claimed is:

1. A method for visualizing an environment of a utility vehicle equipped with a front loader via a visualization system, comprising:

generating first image data using a first camera mounted on a front region of the utility vehicle in front of the cab of the vehicle;

generating second image data using a second camera mounted on the cab;

processing the first and second image data using a data processing unit; and

optically displaying the processed first and second image data using a display unit.

2. The method of claim 1, wherein one of the processed first image data of only the first camera, the processed second image data of only the second camera, and the processed first and second image data of both cameras simultaneously is selectively displayed on the display unit.

3. The method of claim 1, wherein via the data processing unit an actual value of a physical quantity of the front loader is ascertained and optically displayed on the display unit.

4. The method of claim 3, wherein the actual value of the physical quantity is determined as a function of at least one of the following features:

an optical marker of the front loader detected by at least one camera, and

known kinematics data of the front loader.

5. The method of claim 3, wherein a predetermined setpoint value of the physical quantity is displayed and can be entered on the display unit.

6. The method of claim 1, wherein the visualization system is connected via a data bus of the utility vehicle to a controller for the movement control of the front loader.

7. The method of claim 1, wherein via the visualization system an object surrounding the utility vehicle is detected and, depending on the object detected, auxiliary information is derived, which is displayed on the display unit.

8. The method of claim 7, wherein via the data processing unit, an expected future direction of movement of the loading tool toward the object is determined and the auxiliary information is derived from this direction of movement.

9. The method of claim 7, wherein the auxiliary information contains at least one trajectory line, which represents an expected future trajectory of the loading tool and is displayed on the display unit.

10. The method of claim 7, wherein the auxiliary information contains a collision line, which represents an expected collision region between the loading tool and the object and is displayed on the display unit.

11. A utility vehicle having a front loader and a visualization system, comprising:

a first camera mounted on a front region of the utility vehicle in front of the cab of the vehicle and configured to generate first image data;

a second camera mounted on the cab and configured to generate second image data;

a data processing unit configured to process the first and second image data; and

a display unit configured to display the processed first and second image data.

12. The utility vehicle of claim 11, wherein one of the processed first image data of only the first camera, the processed second image data of only the second camera, and the processed first and second image data of both cameras simultaneously is selectively displayed on the display unit.

13. The utility vehicle of claim 11, wherein via the data processing unit an actual value of a physical quantity of the front loader is ascertained and optically displayed on the display unit.

14. The utility vehicle of claim 13, wherein the actual value of the physical quantity is determined as a function of at least one of the following features:

an optical marker of the front loader detected by at least one camera, and

known kinematics data of the front loader.

15. The utility vehicle of claim 13, wherein a predetermined setpoint value of the physical quantity is displayed and can be entered on the display unit.

16. The utility vehicle of claim 11, wherein the visualization system is connected via a data bus of the utility vehicle to a controller for the movement control of the front loader.

17. The utility vehicle of claim 11, wherein via the visualization system an object surrounding the utility vehicle is detected and, depending on the object detected, auxiliary information is derived, which is displayed on the display unit.

18. The utility vehicle of claim 17, wherein via the data processing unit, an expected future direction of movement of the loading tool toward the object is determined and the auxiliary information is derived from this direction of movement.

19. The utility vehicle of claim 17, wherein the auxiliary information contains at least one trajectory line, which represents an expected future trajectory of the loading tool and is displayed on the display unit.

20. The utility vehicle of claim 17, wherein the auxiliary information contains a collision line, which represents an expected collision region between the loading tool and the object and is displayed on the display unit.

21. The utility vehicle of claim 11, wherein the first camera is mounted on an engine hood of the utility vehicle, and the second camera is mounted on the roof of the cab of the utility vehicle.

22. The utility vehicle of claim 11, wherein the first camera has an opening angle of approximately 120°, and the second camera has an opening angle of approximately 90°.

Resources

Images & Drawings included:

Sources:

Recent applications in this class: