CONSTRUCTION SUPPORT SYSTEM

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims priority to Japanese Patent Application No. 2024-203810, filed on Nov. 22, 2024, the entire contents of which are incorporated herein by reference.

BACKGROUND

1. Technical Field

The present disclosure relates to a construction support system.

2. Description of Related Art

Related art discloses that, when a work machine is remotely operated, a mesh-like line diagram (grid) in accordance with distance information is superimposed on the ground and displayed.

SUMMARY

A construction support system according to the present disclosure includes: a photographing device configured to photograph an image of surroundings of a work machine; a display device configured to display an image photographed by the photographing device; and a control device configured to display the image photographed by the photographing device on the display device, and superimpose, on the image photographed by the photographing device, height information indicating a height of a work target upon displaying the image photographed by the photographing device on the display device. The control device is configured to display or hide the height information in accordance with a difference between the height of the work target and a height of a reference plane that is set.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side view of an excavator;

FIG. 2 is a block diagram illustrating a configuration example of a drive system of the excavator illustrated in FIG. 1;

FIG. 3 is a block diagram illustrating a configuration of a controller;

FIG. 4 is a diagram illustrating an embodiment of a construction support system;

FIG. 5 is a flowchart for describing an image display process in the construction support system;

FIG. 6 is a diagram illustrating an example of a display mode of an image;

FIG. 7 is a diagram illustrating another example of the display mode of the image; and

FIG. 8 is a diagram illustrating still another example of the display mode of the image.

DETAILED DESCRIPTION

When the grid is superimposed and displayed on the ground and objects placed on the ground, an operator does not easily see the ground and objects, and thus does not easily work, which is an issue to address.

Therefore, it is preferable to provide a system configured to cause a work site to be easily seen while displaying height information indicating the heights of the ground and the objects.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The embodiments described below do not limit the present disclosure but are illustrative. All of the features described in the embodiments and combinations of the features are not necessarily essential to the present disclosure. Throughout the drawings, the same or corresponding components are denoted by the same or corresponding signs, and descriptions thereof may be omitted.

Work Machine

First, a work machine used in a construction support system of the present disclosure will be described.

FIG. 1 illustrates an excavator 100 as a work machine according to an embodiment of the present disclosure. An upper slewing body 3 is slewably mounted on a lower traveling body 1 of the excavator 100 via a slewing mechanism 2. A boom 4 is attached to the upper slewing body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6, serving as an end attachment, is attached to the tip of the arm 5.

The boom 4, the arm 5, and the bucket 6 form an excavating attachment, which is an example of an attachment. The boom 4 is driven by a boom cylinder 7, the arm 5 is driven by an arm cylinder 8, and the bucket 6 is driven by a bucket cylinder 9.

A boom angle sensor S1 is attached to the boom 4, an arm angle sensor S2 is attached to the arm 5, and a bucket angle sensor S3 is attached to a bucket link. A slewing angle velocity sensor S4 is attached to the upper slewing body 3.

The boom angle sensor S1 is one of the posture detection sensors, and is attached to the boom 4. The boom angle sensor S1 is configured to detect an elevation angle of the boom 4 with respect to the upper slewing body 3, e.g., an angle that is formed, in a side view, by a straight line connecting the fulcrums at both ends of the boom 4 with respect to a slewing plane of the upper slewing body 3. In the present embodiment, the boom angle sensor S1 is an inertial measurement unit (IMU), but may be any other sensor, such as a stroke sensor or the like.

The arm angle sensor S2 is one of the posture detection sensors, and is attached to the arm 5. The arm angle sensor S2 is configured to detect a rotation angle of the arm 5 with respect to the boom 4, e.g., an angle that is formed, in a side view, by a straight line connecting the fulcrums at both ends of the arm 5 with respect to a straight line connecting the fulcrums at both ends of the boom 4. In the present embodiment, the arm angle sensor S2 is an inertial measurement unit (IMU), but may be any other sensor, such as a stroke sensor or the like.

The bucket angle sensor S3 is one of the posture detection sensors, and is attached to the bucket 6. The bucket angle sensor S3 is configured to detect a rotation angle of the bucket 6 with respect to the arm 5, e.g., an angle that is formed, in a side view, by a straight line connecting the fulcrum and the end (cutting end) of the bucket 6 with respect to a straight line connecting the fulcrums at both ends of the arm 5. In the present embodiment, the bucket angle sensor S3 is an inertial measurement unit (IMU), but may be any other sensor, such as a stroke sensor or the like.

Each of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 may be a rotary encoder, an acceleration sensor, a potentiometer (variable resistor), a tilt sensor, an inertial measurement unit, or the like. The inertial measurement unit may be, for example, a combination of an acceleration sensor and a gyro sensor.

The slewing angle velocity sensor S4 is configured to detect the slewing angle velocity of the upper slewing body 3. In the present embodiment, the slewing angle velocity sensor S4 is a gyro sensor. The slewing angle velocity sensor S4 may be configured to calculate a slewing angle based on the slewing angle velocity. The slewing angle velocity sensor S4 may be any other sensor, such as a rotary encoder or the like.

The upper slewing body 3 includes a cab 10, serving as a driver's room, an engine 11, a positioning device 18, a sound collecting device A1, a photographing device C1, a space recognition device 70, a communication device T1, and the like. The cab 10 includes a controller 30. Also, the cab 10 includes a driver's seat, an operation device, and the like. However, the excavator 100 may be an unmanned excavator in which the cab 10 is not provided.

The lower traveling body 1 is not limited to a lower traveling body using crawlers as illustrated, and may be of a wheel (tire) excavator type.

The engine 11 is a drive power source of the excavator 100. In the present embodiment, the engine 11 is a diesel engine. An output shaft of the engine 11 is coupled to input shafts of a main pump 14 (see FIG. 2) and a pilot pump 15 (see FIG. 2).

The positioning device 18 is configured to measure a position of the excavator 100. In the present embodiment, the positioning device 18 is a global navigation satellite system (GNSS) compass, and is configured to measure the position and orientation of the upper slewing body 3. The positioning device 18 transmits data of the position and orientation of the excavator 100 to the controller 30.

The sound collecting device A1 is configured to collect sounds generated around the excavator 100. In the present embodiment, the sound collecting device A1 is a microphone attached to the upper slewing body 3.

The photographing device C1 is configured to photograph the surroundings of the excavator 100. In the present embodiment, the photographing device C1 includes a rear camera C1B attached to the rear end of the upper surface of the upper slewing body 3, a front camera C1F attached to the front end of the upper surface of the cab 10, a left camera C1L attached to the left end of the upper surface of the upper slewing body 3, and a right camera C1R attached to the right end of the upper surface of the upper slewing body 3. The photographing device C1 may be an omnidirectional camera disposed at a predetermined position in the cab 10. The predetermined position is, for example, a position corresponding to the position of the eyes of an operator sitting on the driver's seat provided in the cab 10. The photographing device C1 is a monocular camera having the function of adjusting a focus and a depth of field in an optical system.

The space recognition device 70 is configured to recognize an object existing in a three-dimensional space around the excavator 100, and measure (calculate) a positional relationship, such as, for example, a distance from the space recognition device 70 or the excavator 100 to the recognized object. The space recognition device 70 can include, for example, an ultrasonic sensor, a millimeter-wave radar, a monocular camera, a stereo camera, a light detecting and ranging (LIDAR) sensor, a range image sensor, an infrared sensor, or the like. The space recognition device 70 acquires, for example, recess and projection information of the ground in a photographing range of the front camera C1F.

The communication device T1 is configured to control communication with a device outside the excavator 100. In the present embodiment, the communication device T1 is configured to control wireless communication between the communication device T1 and a device outside the excavator 100 via a wireless communication network.

The controller 30 is an example of a control device according to the present disclosure, and is an arithmetic unit configured to execute various calculations. In the present embodiment, the controller 30 includes a microcomputer including a CPU and a memory. Various functions of the controller 30 are realized by the CPU executing a program stored in the memory.

FIG. 2 is a block diagram illustrating a configuration example of a drive system of the excavator 100 illustrated in FIG. 1. In FIG. 2, a mechanical power transmission line is indicated by a double line, a hydraulic oil line is indicated by a thick solid line, a pilot line is indicated by a broken line, and an electric control line is indicated by a dotted line.

The drive system of the excavator 100 includes the engine 11, a regulator 13, the main pump 14, the pilot pump 15, a control valve unit 17, the controller 30, an electromagnetic valve unit 19, and the like. Driving of the engine 11 is controlled by an engine control unit 74.

The main pump 14 is configured to supply hydraulic oil to the control valve unit 17 through a hydraulic oil line 16. In the present embodiment, the main pump 14 is a swashplate-type variable displacement hydraulic pump.

The regulator 13 is configured to control the discharge amount of the main pump 14. In the present embodiment, the regulator 13 is configured to adjust the tilt angle of the swashplate of the main pump 14 in accordance with the discharge pressure of the main pump 14, a control signal from the controller 30, or the like. The discharge amount per one rotation (displacement) of the main pump 14 is controlled by the regulator 13.

The pilot pump 15 is configured to supply hydraulic oil to various hydraulic control devices through the pilot line 25. In the present embodiment, the pilot pump 15 is a fixed-displacement hydraulic pump. However, the pilot pump 15 may be omitted. In this case, the function of the pilot pump 15 may be realized by the main pump 14. That is, the main pump 14 may have the function of supplying hydraulic oil to the electromagnetic valve unit 19 or the like via a throttle or the like, in addition to the function of supplying hydraulic oil to the control valve unit 17.

The control valve unit 17 is configured to selectively supply hydraulic oil received from the main pump 14 to one or more hydraulic actuators. In the present embodiment, the control valve unit 17 includes a plurality of control valves corresponding to a plurality of hydraulic actuators. The control valve unit 17 is configured to selectively supply hydraulic oil, discharged from the main pump 14, to one or more hydraulic actuators. The hydraulic actuator includes, for example, the boom cylinder 7, the arm cylinder 8, the bucket cylinder 9, a left traveling hydraulic motor 1L, a right traveling hydraulic motor 1R, and a slewing hydraulic motor 2A.

The controller 30 is configured to control the electromagnetic valve unit 19 based on an operation signal received via the communication device T1. In the present embodiment, the operation signal is transmitted from a remote operation room. The operation signal may be generated by the operation device provided in the cab 10.

The electromagnetic valve unit 19 includes a plurality of electromagnetic valves disposed at each pilot line 25 connecting the pilot pump 15 and a pilot port of each control valve in the control valve unit 17.

In the present embodiment, the controller 30 individually controls the opening area of each of a plurality of electromagnetic valves, thereby enabling control of the pilot pressure applied to the pilot port of each control valve. Therefore, the controller 30 can control the flow rate of hydraulic oil flowing into each hydraulic actuator and the flow rate of hydraulic oil flowing out from each hydraulic actuator, and thus can control the movement of each hydraulic actuator.

In this manner, the controller 30 can realize raising and lowering of the boom 4, opening and closing of the arm 5, opening and closing of the bucket 6, slewing of the upper slewing body 3, traveling of the lower traveling body 1, and the like, in accordance with operation signals from the outside, such as the remote operation room or the like.

The excavator 100 may have a configuration in which some or all of the driven elements, such as the lower traveling body 1, the upper slewing body 3, the boom 4, the arm 5, and the bucket 6 are electrically driven by electric actuators. That is, the excavator 100 may be a hybrid excavator or an electric excavator.

Configuration of Controller 30

FIG. 3 is a block diagram illustrating a configuration of the controller 30.

As illustrated in FIG. 3, the controller 30 includes an image acquisition part 31, a recess and projection information acquisition part 32, a height information generation part 33, a reference plane information acquisition part 34, a height display information generation part 35, and a display control part 36.

The image acquisition part 31 is configured to acquire an image photographed by the front camera C1F of the photographing device C1. Thus, the image acquisition part 31 acquires an image including the ground in front of the excavator 100.

The recess and projection information acquisition part 32 is configured to acquire recess and projection information of the ground acquired by the space recognition device 70. Thus, the recess and projection information acquisition part 32 acquires recess and projection information of the ground in front of the excavator 100.

The height information generation part 33 is configured to generate grid information based on the recess and projection information of the ground acquired by the recess and projection information acquisition part 32. The grid information is an example of height information in the present disclosure. The grid information is a lattice-shaped piece of information displayed on a display device D1, and is represented by a grid in which the lattice shape changes in accordance with a gradient of the ground. For example, the grid information may be a lattice formed by intersection between: longitudinal lines extending in a longitudinal direction, i.e., a front-rear direction of the excavator 100 (an X-axis direction in FIG. 1); and transverse lines extending in a transverse direction, i.e., a lateral direction of the excavator 100 (a direction perpendicular to an XZ plane in FIG. 1). The longitudinal lines and the transverse lines are straight lines in a region in which the height of the ground does not change. When the height of the ground changes, the longitudinal lines and the transverse lines are curved or bent in accordance with a change in the height of the ground. Further, the gap between the longitudinal lines may be wider as the longitudinal lines are closer to a viewer, thereby providing the viewer with a sense of perspective. Therefore, the recesses and projections of the ground can be recognized in accordance with the shape of the lattice.

The reference plane information acquisition part 34 is configured, for example, to acquire information indicating the height and tilt of a design target plane. The position of the excavator 100 is measured by the positioning device 18. Therefore, when construction information of the excavator 100 is set in the controller 30, the reference plane information acquisition part 34 can acquire information indicating the height and tilt of the design target plane.

The height display information generation part 35 is configured to generate, in the grid information generated by the height information generation part 33, grid display information in which a portion corresponding to the height of the reference plane acquired by the reference plane information acquisition part 34 is hidden.

The display control part 36 is configured to superimpose the grid display information generated by the height display information generation part 35 on the image acquired by the image acquisition part 31, and display the superimposed image on the display device D1 of a remote operation room RC (see FIG. 4), which will be described below.

Configuration of Construction Support System

Next, a construction support system using the excavator 100 in the present embodiment will be described.

FIG. 4 is a diagram illustrating an embodiment of the construction support system of the present disclosure.

As illustrated in FIG. 4, a construction support system SYS in the present embodiment includes the excavator 100 and the remote operation room RC in which the excavator 100 is operated remotely.

In the construction support system SYS configured in this manner, the communication device T1 of the excavator 100 is configured to perform, using wireless communication, transmission and reception of information to and from a communication device T2 disposed in the remote operation room RC. In the present embodiment, the communication device T1 and the communication device T2 are configured to perform transmission and reception of information through the 5^th generation mobile communication line (5G line), an LTE line, a satellite line, or the like.

The remote operation room RC includes a remote controller 40, a sound output device A2, an indoor photographing device C2, the display device D1, the communication device T2, and the like. Also, the remote operation room RC includes a driver's seat DS for an operator OP who remotely operates the excavator 100.

The remote controller 40 is an arithmetic unit configured to execute various calculations. In the present embodiment, similar to the controller 30, the remote controller 40 includes a microcomputer including a CPU and a memory. Various functions of the remote controller 40 are realized by the CPU executing a program stored in the memory.

The sound output device A2 is configured to output a sound. In the present embodiment, the sound output device A2 is a speaker and configured to reproduce sounds collected by the sound collecting device A1 attached to the excavator 100.

The indoor photographing device C2 is configured to photograph the interior of the remote operation room RC. In the present embodiment, the indoor photographing device C2 is a camera disposed in the remote operation room RC, and is configured to photograph the operator OP sitting on the driver's seat DS.

The communication device T2 is configured to control wireless communication with the communication device T1 attached to the excavator 100.

In the present embodiment, the driver's seat DS has the same structure as the driver's seat disposed in the cab of a typical excavator. Specifically, a left console box is disposed on the left side of the driver's seat DS, and a right console box is disposed on the right side of the driver's seat DS. A left operation lever is disposed at the front end of the upper surface of the left console box, and a right operation lever is disposed at the front end of the upper surface of the right console box. Also, a traveling lever and a traveling pedal are disposed in front of the driver's seat DS. Further, an engine revolution speed adjustment dial 75 is disposed at the center of the upper surface of the right console box. The left operation lever, the right operation lever, the traveling lever, the traveling pedal, and the engine revolution speed adjustment dial 75 form an operation device 26.

The engine revolution speed adjustment dial 75 is a dial configured to adjust the engine revolution speed of the engine 11, and is configured, for example, to switch the engine revolution speed in four steps.

Specifically, the engine revolution speed adjustment dial 75 is configured to switch the engine revolution speed in the following four steps: an SP mode, an H mode, an A mode, and an idling mode. The engine revolution speed adjustment dial 75 transmits data of settings of the engine revolution speed to the controller 30.

The SP mode is a revolution speed mode selected when the operator OP wishes to give priority to work efficiency, and the SP mode uses the highest engine revolution speed. The H mode is a revolution speed mode selected when the operator OP wishes to strike a balance between work efficiency and fuel efficiency, and the H mode uses the second highest engine revolution speed. The A mode is a revolution speed mode selected when the operator OP wishes to operate the excavator with low noise while giving priority to fuel efficiency, and the A mode uses the third highest engine revolution speed. The idling mode is a speed mode selected when the operator OP wishes to achieve an idling state of the engine, and the idling mode uses the lowest engine revolution speed. The revolution speed of the engine 11 is controlled to be constant at the engine revolution speed of the revolution speed mode selected using the engine revolution speed adjustment dial 75.

The operation device 26 includes an operation sensor 29 configured to detect operation content of the operation device 26. The operation sensor 29 is, for example, a tilt sensor configured to detect a tilt angle of the operation lever or an angle sensor configured to detect a pivot angle about a pivot shaft of the operation lever. The operation sensor 29 may include another sensor, such as a pressure sensor, a current sensor, a voltage sensor, a distance sensor, or the like. The operation sensor 29 outputs, to the remote controller 40, information of the detected operation content of the operation device 26. The remote controller 40 generates an operation signal based on the received information, and transmits the generated operation signal to the excavator 100. The operation sensor 29 may be configured to generate an operation signal. In this case, the operation sensor 29 may output the operation signal to the communication device T2 without the remote controller 40.

The controller 30 of the excavator 100 that received the operation signal realizes raising and lowering of the boom 4, opening and closing of the arm 5, opening and closing of the bucket 6, slewing of the upper slewing body 3, traveling of the lower traveling body 1, and the like, in accordance with the received operation signal.

The display device D1 is configured to display information of the surroundings of the excavator 100. In the present embodiment, the display device D1 is a multi-display including nine monitors, i.e., three by three in a matrix, and is configured to display spaces in front of, leftward of, and rightward of the excavator 100. Each of the monitors is a liquid crystal monitor, an organic EL monitor, or the like. However, the display device D1 may include one or more curved monitors, or may be a projector.

The display device D1 is configured to display an image for enabling the operator OP in the remote operation room RC to visually recognize the surroundings of the excavator 100. That is, the display device D1 displays the image such that the operator can confirm the surroundings of the excavator 100 as if the operator were in the cab 10 of the excavator 100 even though the operator is in the remote operation room RC.

Also, the remote operation room RC may include an input device 27. The input device 27 is configured to allow the operator OP sitting on the driver's seat DS to input information. The input information may be transmitted to the controller 30 of the excavator 100 via the communication device T2. Also, the operator OP can set a reference plane using the input device 27. The reference plane is used for hiding a portion of the grid information generated by the height information generation part 33. For example, it is possible to set a reference plane different from the design target plane by displaying the design target plane on the display device D1 and then raising or lowering the height of the reference plane or changing the tilt of the reference plane in accordance with an input to the input device 27. Thus, the reference plane information acquisition part 34 acquires information indicating the height and tilt of the reference plane set using the input device 27. Then, the height display information generation part 35 generates grid display information in which a portion of the grid information, generated by the height information generation part 33, corresponding to the height of the reference plane acquired by the reference plane information acquisition part 34 is hidden.

The display device D1 may be a display device configured to be worn by the operator OP. For example, the display device D1 may be a head mounted display (HMD), and may be configured to perform transmission and reception of information to and from the remote controller 40 through wireless communication. The head mounted display may be connected by wired to the remote controller 40. The head mounted display may be a transmissive head mounted display or a non-transmissive head mounted display. The head mounted display may be a monocular-type head mounted display or a binocular-type head mounted display. As described above, when the display device D1 is a display device configured to be worn by the operator OP, a gaze measuring device 28 may be provided in the display device D1.

For ease of understanding, the present embodiment has been described taking, as an example, a configuration including the single excavator 100 and the single remote operation room RC. However, a plurality of the excavators 100 and a plurality of the remote operation rooms RC may be provided. In this case, the plurality of the remote operation rooms RC may be caused to respectively correspond to the plurality of the excavators 100. Alternatively, the plurality of the excavators 100 may be operated in the single remote operation room RC.

Image Display Process

In the following, an image display process in the construction support system SYS configured as described above will be described below.

FIG. 5 is a flowchart for describing the image display process in the construction support system SYS.

The surroundings of the excavator 100 are photographed by the photographing device C1 of the excavator 100. The front camera C1F of the photographing device C1 is attached to the front end of the upper surface of the cab 10. Thus, the image in front of the excavator 100 is photographed along with the ground in front of the excavator 100.

The image photographed by the front camera C1F is transmitted to the controller 30, and acquired by the image acquisition part 31 (step ST11). Thus, the image acquisition part 31 acquires the image photographed by the front camera C1F and including the ground in front of the excavator 100.

The recess and projection information of the ground can be acquired by the space recognition device 70 of the excavator 100. The space recognition device 70 is configured, for example, to apply laser light to the ground in front of the excavator 100 and receive reflected light, thereby acquiring the recess and projection information of the ground. Specifically, when a LIDAR sensor is used as the space recognition device 70, laser light is applied to a photographing range of the front camera C1F. Then, point cloud data of distances determined from the reflected light is used to acquire the recess and projection information of the ground in the photographing range of the front camera C1F. The recess and projection information of the ground acquired by the space recognition device 70 is transmitted to the controller 30, and acquired by the recess and projection information acquisition part 32 (step ST12). Thus, the recess and projection information acquisition part 32 acquires the recess and projection information of the ground in front of the excavator 100, i.e., the ground acquired by the image acquisition part 31 and included in the image photographed by the front camera C1F. Here, when the LIDAR sensor is used as the space recognition device 70 to acquire the recess and projection information of the ground by the LIDAR sensor, the laser light applied from the LIDAR sensor may be reflected by the attachments, i.e., the boom 4, the arm 5, and the bucket 6. In this case, it is not possible to accurately acquire the recess and projection information of the ground using the reflected light of the laser light applied from the LIDAR sensor. Thus, the recess and projection information acquisition part 32 may acquire information of the posture of the boom 4 detected by the boom angle sensor S1, the posture of the arm 5 detected by the arm angle sensor S2, and the posture of the bucket 6 detected by the bucket angle sensor S3. In this case, the recess and projection information acquisition part 32 can recognize the postures of the boom 4, the arm 5, and the bucket 6. As a result, the recess and projection information acquisition part 32 can exclude the reflected light reflected by the attachments of the boom 4, the arm 5, and the bucket 6 from the reflected light for acquiring the recess and projection information of the ground in the reflected light of the laser light applied from the LIDAR sensor, and can accurately acquire the recess and projection information of the ground.

When the recess and projection information of the ground acquired by the space recognition device 70 is acquired by the recess and projection information acquisition part 32, the height information generation part 33 generates grid information based on the recess and projection information of the ground acquired by the recess and projection information acquisition part 32 (step ST13). The grid information is a lattice-shaped piece of information formed of longitudinal lines and transverse lines, and the shape of the lattice changes in accordance with the gradient of the ground. Specifically, in a flat region of the ground, the transverse lines are straight lines parallel to each other, and the longitudinal lines are straight lines in which the gap is wider as the longitudinal lines are closer to a viewer for providing the viewer with a sense of perspective, as described above. In a region having a gradient differing from the flat region of the ground, when the gradient is constant, either or both of the transverse lines and the longitudinal lines are straight lines having a constant angle with respect to the flat region of the ground. Change in the angle of either or both of the transverse lines and the longitudinal lines enables recognition of a gradient with respect to the flat region of the ground. In addition, the extent of the gradient can be recognized based on the magnitude of the angle. Also, when the gradient changes, the transverse lines or the longitudinal lines are curved lines in accordance with the change in the gradient. It is therefore possible to recognize how the gradient changes in accordance with the shape of the curved lines.

In this manner, the height information may be the grid information represented by the grid in which the shape of the lattice changes in accordance with the gradient of the ground. Since the grid information is typically used for displaying the recess and projection information, it is possible to easily understand the recesses and projections of the ground.

In the present embodiment, the height information is information in the lattice shape represented by the longitudinal lines and the transverse lines. However, the height information may be concentric circles or may be fan shapes having lines extending in the radial direction from the slewing center of the excavator 100. Alternatively, the height may be indicated using a color. In this manner, since the height information is height information indicated by a figure or a color, it is possible to easily understand the recesses and projections of the ground compared to the height information being indicated by characters.

Also, for example, the reference plane information acquisition part 34 acquires, as reference plane information, information indicating the height and tilt of the design target plane (step ST14). As described above, since the position of the excavator 100 is measured by the positioning device 18, the reference plane information acquisition part 34 can acquire information indicating the height and tilt of the design target plane using the construction information in which the height of the design target plane is set.

When the grid information is generated by the height information generation part 33 and the reference plane information indicating the height of the reference plane is acquired by the reference plane information acquisition part 34, the height display information generation part 35 processes the grid information generated by the height information generation part 33, and generates the grid display information in which, of the grid information generated by the height information generation part 33, a portion corresponding to the height of the reference plane acquired by the reference plane information acquisition part 34 is hidden (step ST15). An example of the grid display information generated is grid display information in which, of the grid information generated by the height information generation part 33, the grid information of a portion higher than the reference plane acquired by the reference plane information acquisition part 34 is hidden. Here, the grid information generated by the height information generation part 33 is based on the recess and projection information of the ground acquired by the recess and projection information acquisition part 32, and the recess and projection information of the ground is acquired, for example, using the point cloud data of the distances between the LIDAR sensor and the ground acquired by the LIDAR sensor when the LIDAR sensor is used as the space recognition device 70. Therefore, by using the disposition position (height) of the LIDAR sensor and the point group data, it is possible to understand the height of each region of the grid information generated by the height information generation part 33.

Subsequently, in the display control part 36, the grid display information generated by the height display information generation part 35 is superimposed on the image acquired by the image acquisition part 31 (step ST16). Then, the image information in which the grid display information generated by the height display information generation part 35 is superimposed on the image acquired by the image acquisition part 31 is transmitted to the remote operation room RC via the communication device T1.

The image information transmitted via the communication device T1 is received via the communication device T2 of the remote operation room RC, and is displayed on the display device D1 of the remote operation room RC under control of the display control part 36 (step ST17). Thus, an image in which the grid display information generated by the height display information generation part 35 is superimposed on the image acquired by the image acquisition part 31 is displayed on the display device D1 of the remote operation room RC.

Display Mode of Image

In the following, a display mode of the image displayed through the above-described display process will be described.

<First Display Mode>

FIG. 6 is a diagram illustrating an example of the display mode of the image.

As illustrated in FIG. 6, in the present example, an image including a flat portion 201, a projection 202, and a recess 203 is displayed on the display device D1 of the remote operation room RC. This image is photographed by the front camera C1F and acquired by the image acquisition part 31, i.e., an image including the ground in front of the excavator 100.

The flat portion 201 is a flat plane having the same height as that of the design target plane in the construction information. The projection 202 is a region higher than the flat portion 201 obtained by projecting from the flat portion 201. The recess 203 is a region lower than the flat portion 201 obtained by depressing from the flat portion 201.

In this manner, the grid display information generated by the height display information generation part 35 is superimposed on the image including the flat portion 201, the projection 202, and the recess 203.

Here, the grid display information generated by the height display information generation part 35 does not need to be superimposed on the overall region of the image photographed by the front camera C1F and acquired by the image acquisition part 31. As illustrated in FIG. 6, the grid display information generated by the height display information generation part 35 may be superimposed only on a grid display region 300 that is set in the image including the flat portion 201, the projection 202, and the recess 203. The grid display region 300 may be set, for example, through designation using the input device 27. Here, the grid display region 300 may be designated in a state in which the grid display information generated by the height display information generation part 35 is superimposed on the image acquired by the image acquisition part 31. Alternatively, the grid display region 300 may be previously set. When the grid display region 300 is previously set, for example, the recess and projection information acquisition part 32 may previously set a range in which the recess and projection information of the ground acquired by the space recognition device 70 is to be acquired.

The grid display information generated by the height display information generation part 35 is superimposed on the flat portion 201 as grid lines 301, but is not superimposed on the projection 202.

In this manner, in the present example, the grid lines 301 serving as the grid information are displayed in a region in which the current height of the ground is equal to that of the reference plane, while the grid lines serving as the grid information are hidden in a region in which the current height of the ground is higher than that of the reference plane. As a result, of the ground in front of the excavator 100 photographed by the front camera C1F, the grid information is not displayed in a region that needs to be excavated by the bucket 6, i.e., that region can be easily seen.

The grid display information generated by the height display information generation part 35 is superimposed on the recess 203 as grid lines 302, which are in a display mode different from that of the grid lines 301. For example, the grid lines 301 may be displayed as solid lines, while the grid lines 302 may be displayed as broken lines. Also, the grid lines 301 and the grid lines 302 may be displayed in different colors. Thus, due to the difference in the display mode of the recess 203 in addition to the shape of a lattice of the grid lines 302, it is possible to recognize, for example, that the recess 203 is depressed compared to the flat portion 201 through excessive excavation and needs to be filled with soil.

In this manner, in a region in which the grid information serving as the height information is displayed, the display mode of the grid lines may be different in accordance with the current height of the ground. This enables the height to be easily understood in the region in which the grid information is displayed.

In the present embodiment as described above, the construction support system includes the photographing device C1 configured to photograph the image of the surroundings of the excavator 100, the display device D1 configured to display the image photographed by the photographing device C1, and the controller 30 configured to display the image photographed by the photographing device C1 on the display device D1, and superimpose, on the image photographed by the photographing device C1, the height information indicating the height of the work target upon displaying the image photographed by the photographing device C1 on the display device D1, in which the controller 30 is configured to display or hide the height information in accordance with the difference between the height of the work target and the height of the reference plane that is set. This enables the work site to be easily seen while displaying the height information indicating the height of the ground. Here, when the height information indicating the height of the ground is displayed, a sense of depth or three-dimensionality is obtained, and a state suitable for work is maintained. The height of the work target may be the current height of the ground, as described above.

Also, as described above, the controller 30 may display or hide the height information in a set region of the image photographed by the photographing device C1. Thus, it is possible to display or hide the height information in a region needed to be worked by the excavator 100 in the image photographed by the photographing device C1.

<Second Display Mode>

FIG. 7 is a diagram illustrating another example of the display mode of the image.

In the present example, the mode of the grid display information in the grid display region 300 is different from the example illustrated in FIG. 6.

As illustrated in FIG. 7, in the present example, the grid display information generated by the height display information generation part 35 is superimposed as the grid lines 301 on the upper surface of the projection 202, except for projections 204a and 204b that further project from the upper surface of the projection 202, and is not superimposed on the projections 204a and 204b.

Also, the grid display information generated by the height display information generation part 35 is superimposed on the flat portion 201 and the side surface of the projection 202 as the grid lines 302, which are in a display mode different from that of the grid lines 301. Also, the grid display information generated by the height display information generation part 35 is superimposed on the recess 203 as grid lines 303, which are in a display mode different from those of the grid lines 301 and 302. For example, the grid lines 301 may be displayed as solid lines, the grid lines 302 may be displayed as broken lines, and the grid lines 303 may be displayed as dashed-dotted lines. Also, display colors of the grid lines 301 to 303 may be different from each other.

In this manner, it is possible to change the height of the reference plane, used as a reference for displaying or hiding the grid lines serving as the height information, from the height of the design target plane. For example, as described above, it is possible to set a reference plane different from the design target plane by displaying the design target plane on the display device D1 and then raising or lowering the height of the reference plane or changing the tilt of the reference plane in accordance with an input to the input device 27. In this case, the reference plane information acquisition part 34 acquires information indicating the height and tilt of the reference plane that is set using the input device 27. Then, the height display information generation part 35 generates grid display information in which a portion of the grid information, generated by the height information generation part 33, corresponding to the height of the reference plane acquired by the reference plane information acquisition part 34 is hidden. In this manner, the operator OP can set the reference plane, used for hiding a portion of the grid information generated by the height information generation part 33, using the input device 27.

Also, a flat plane parallel to the bottom surface of the bucket 6 or the reference plane and passing through the toe of the bucket 6 may be set as the reference plane. In this case, for example, first, the bucket 6 is moved to a region desired as the reference plane, and the bottom surface of the bucket 6 or the toe of the bucket 6 is contacted with the region desired as the reference plane. In this state, for example, a message to set the reference plane is input using the input device 27. Here, as described above, the position of the excavator 100 is measured by the positioning device 18. Also, the posture of the boom 4 is detected by the boom angle sensor S1, the posture of the arm 5 is detected by the arm angle sensor S2, and the posture of the bucket 6 is detected by the bucket angle sensor S3. Therefore, the reference plane information acquisition part 34 can acquire information indicating the height and tilt of the reference plane that is set in a state in which the bottom surface of the bucket 6 or the toe of the bucket 6 is in contact with the ground. Also, the plane on which the excavator 100 currently exists may be set as the reference plane.

In this manner, since the height or tilt of the reference plane is changeable, the operator OP can desirably set, in accordance with the intended work, the height at which the grid lines are hidden in the grid display region 300 in which the grid information is to be displayed.

<Third Display Mode>

FIG. 8 is a diagram illustrating still another example of the display mode of the image.

In the present example, the mode of the grid display information in the grid display region 300 is different from the example illustrated in FIG. 6.

As illustrated in FIG. 8, in the present example, the grid display information generated by the height display information generation part 35 is superimposed on the projection 202 as the grid line 301, but is not superimposed on the flat portion 201 having the same height as that of the reference plane. Also, the grid display information generated by the height display information generation part 35 is superimposed on the recess 203 as the grid lines 302, which are in a display mode different from that of the grid lines 301. For example, the grid lines 301 may be displayed as solid lines, while the grid lines 302 may be displayed as broken lines. Also, display colors of the grid lines 301 and the grid lines 302 may be different from each other.

In this manner, in the present example, the grid lines are hidden only on the flat portion 201 of the grid display region 300 that has the same height as that of the design target plane. That is, in a region in which the current height of the ground is equal to the height of the reference plane, the grid line serving as the height information is hidden. With this configuration, it is possible to recognize the recesses and projections of a region needed to be worked in a manner that is clearly distinguishable from the design target plane.

For a portion in which the grid line is to be displayed, the display mode of the grid lines may be different in accordance with a difference in height from the reference plane. For example, the grid lines of a portion in which the absolute value of a difference in height from the reference plane is less than 100 millimeters (mm) may be displayed in blue, and the grid lines of a portion in which the absolute value of a difference in height from the reference plane is 100 mm or more may be displayed in red. Also, the grid lines of the portion in which the absolute value of the difference in height from the reference plane is less than 100 mm may be displayed as solid lines, and the grid lines of the portion in which the absolute value of the difference in height from the reference plane is 100 mm or more may be displayed as broken lines.

In the above-described embodiments, the recess and projection information of the ground is acquired using the space recognition device 70, such as a LIDAR sensor or the like. However, the way to acquire the recess and projection information of the ground is not limited to the way in which the space recognition device 70 is used. For example, the recess and projection information of the ground may be acquired from a trajectory of the bucket 6 during excavation. In this case, the recess and projection information of the ground acquired from the trajectory of the bucket 6 can be shared between the plurality of the excavators 100 or utilized as a work history in subsequent excavation work.

Also, in the above-described embodiments, the height information is displayed or hidden in accordance with the difference between the current height of the ground and the height of the reference plane. However, the height information may be displayed or hidden in accordance with the difference between the height of the object placed on the reference plane and the height of the reference plane. Examples of the object placed on the reference plane include wood, metals for recycling (metal scraps), cones, signboards, and the like. Also, the object placed on the reference plane includes objects partially buried in the reference plane, such as a fence and the like.

In the above-described embodiments, the controller 30 included in the excavator 100 is an example of the control device of the present disclosure. However, the remote controller 40 may function as an example of the control device of the present disclosure. Alternatively, the function of the controller 30 illustrated in FIG. 3 may be realized on the cloud.