WORK VEHICLE PATH PLAN GENERATION SYSTEM AND WORK VEHICLE PATH PLAN GENERATION METHOD

Description

TECHNICAL FIELD

The present disclosure relates to a work vehicle path plan generation system and a work vehicle path plan generation method.

The present application claims priority with respect to Japanese Patent Application No. 2022-106372 filed in Japan on Jun. 30, 2022, the contents of which are incorporated herein by reference.

BACKGROUND ART

In an excavation plan creation device disclosed in Patent Document 1, a machine learning model is used in which topographic information is input and planned values of an excavation trajectory and a swing direction are output. This excavation plan creation device generates a plurality of plan models subjected to machine learning by changing each parameter related to soil quality with an excavation efficiency as an evaluation value. Further, this excavation plan creation device estimates the soil quality, selects the plan model based on the estimated soil quality, inputs the topographic information to the selected plan model, and calculates the planned value as an output of the plan model. In addition, this excavation plan creation device estimates the topographic information based on time-series data of a teeth position of a bucket and a load of work equipment that supports the bucket, and the estimated soil quality.

CITATION LIST

Patent Document

[Patent Document 1]

• • Japanese Unexamined Patent Application, First Publication No. 2021-188362

SUMMARY OF INVENTION

Technical Problem

In the excavation plan creation device disclosed in Patent Document 1, it is possible to create an excavation plan with high accuracy in consideration of the influence of the soil quality appropriately in a case where the excavation plan is created. On the other hand, for example, it is necessary to perform estimation calculation or the like based on the time-series data of the teeth position of the bucket and the load of the work equipment that supports the bucket, and the soil quality. In particular, in a case where an earth is handled as a particle model for the soil quality, there is a possibility that a load of calculation processing may be increased.

The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide a work vehicle path plan generation system and a work vehicle path plan generation method capable of efficiently generating a path plan for the work vehicle.

Solution to Problem

In order to achieve the aforementioned objects, one aspect of the present disclosure provides a work vehicle path plan generation system that generates a path plan in order for a work vehicle including work equipment to perform an excavation work on a ground of a construction target area. The work vehicle path plan generation system includes a position detection unit configured to detect a position of the work vehicle, an information storage unit configured to store topographic shape information indicating a shape of a topography in the construction target area, a position of the work vehicle, and design surface information indicating a target shape in the construction target area, and a path plan generation unit configured to generate a work equipment path plan indicating a movement path of the work equipment and a travel path plan indicating a travel path of the work vehicle, based on the topographic shape information, the position of the work vehicle, and the design surface information.

Advantageous Effects of Invention

According to each aspect of the present disclosure, it is possible to efficiently generate the path plan of the work vehicle.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 A plan view of a construction target area in which an excavation work is performed by a work vehicle according to an embodiment.

FIG. 2 A side-sectional view of the construction target area in which the excavation work is performed by the work vehicle according to an embodiment.

FIG. 3 A schematic block diagram showing a configuration example of a path plan generation device according to an embodiment.

FIG. 4 A diagram showing an example of a movement path of work equipment in a travel path of the work vehicle according to an embodiment.

FIG. 5 A diagram showing an example of a simulation model used for a simulation for candidates for the movement path in a path plan generation unit according to an embodiment.

FIG. 6 A diagram showing another example of the movement path of the work equipment in the travel path of the work vehicle according to an embodiment.

FIG. 7 A diagram showing still another example of the movement path of the work equipment in the travel path of the work vehicle according to an embodiment.

FIG. 8 A side view of an earth held in front of a blade and an earth of windrows formed by overflowing onto both sides of the blade, in a case where excavation is performed by the work equipment according to an embodiment.

FIG. 9 A plan view of the earth held in front of the blade and the earth of the windrows formed by overflowing onto both sides of the blade, in a case where the excavation is performed by the work equipment according to an embodiment.

FIG. 10 A diagram in which the earth held in front of the blade and the earth of the windrows formed by overflowing onto both sides of the blade, in a case where the excavation is performed by the work equipment according to an embodiment, are modeled as a polygonal prism-shaped three-dimensional model.

FIG. 11 A flowchart showing an operation of the path plan generation device according to an embodiment.

FIG. 12 A diagram showing an outline of a flow of generating a path plan with learning by reinforcement learning according to an embodiment.

FIG. 13 A schematic block diagram showing a configuration example of a work vehicle path plan generation system according to an embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present disclosure will be described with reference to drawings. In each drawing, the same reference numerals are used for the same or corresponding configurations, and the description thereof will be omitted as appropriate.

(Configuration Example of Work Vehicle 100)

FIG. 1 is a plan view of a construction target area A in which an excavation work is performed by a work vehicle 100 (work machine) according to the embodiment. FIG. 2 is a side-sectional view of the construction target area A in which the excavation work is performed by the work vehicle 100 according to the embodiment.

As shown in FIGS. 1 and 2, in the present embodiment, the work vehicle 100 performs the excavation work on a ground G of the predetermined construction target area A. The work vehicle 100 excavates the ground G to form an excavated ground surface K along a design surface S designed in advance. The construction target area A shown in FIGS. 1 and 2 is merely an example, and a planar shape and the like thereof can be changed as appropriate. In addition, a topographic shape of a ground surface of the ground G and a shape of the design surface S (excavated ground surface K) are merely examples and can be changed as appropriate.

The work vehicle 100 is automatically operated by a remote operation at a construction site including the construction target area A to excavate the ground G. The work vehicle 100 according to the embodiment is a bulldozer as an example. The work vehicle 100 includes an undercarriage 110, an upper vehicle body 120, and work equipment 130.

The undercarriage 110 supports the work vehicle 100 in a travelable manner. The undercarriage 110 includes, for example, a pair of left and right crawlers of a crawler 110a (also referred to as left crawler 110a) and a crawler 110b (also referred to as right crawler 110b). Both left and right crawlers 110a and 110b can be driven independently drive wheels to move forward and move rearward. In a case where the left crawler 110a and the right crawler 110b are moved forward at the same time, the undercarriage 110 moves forward. In a case where the left crawler 110a and the right crawler 110b are moved rearward at the same time, the undercarriage 110 moves rearward. In addition, in a case where the drive wheel of one crawler and the drive wheel of the other crawler are driven in opposite directions to each other, for example, in a case where the right crawler 110b is moved forward and the left crawler 110a is moved rearward at the same time, the undercarriage 110 can rotate around a swing center.

The upper vehicle body 120 is supported on the undercarriage 110. The upper vehicle body 120 includes a cab 121. The cab 121 is a space where an operator (driver) boards to operate the work vehicle 100.

The work equipment 130 includes at least a lift frame 131 and a blade 133. The lift frame 131 is attached to the undercarriage 110 in an operable manner. The blade 133 excavates an earth or the like. The blade 133 is attached to the lift frame 131 in an operable manner.

As shown in FIG. 1, the work vehicle 100 excavates the ground G with the blade 133 while moving along a plurality of travel paths R in the construction target area A. In FIG. 1, each of the plurality of travel paths R is linear in a plan view. However, the travel paths R are not limited to being linear, and may be curved or bent as appropriate in accordance with topography, an obstacle, or the like. In addition, in FIG. 1, the plurality of travel paths R are set to be parallel to each other in a plan view, but may be, for example, extended in a radial shape.

(Configuration Example of Path Plan Generation Device 20)

FIG. 3 is a schematic block diagram showing a configuration example of a path plan generation device 20 according to the embodiment.

As shown in FIG. 3, the path plan generation device 20 can be configured by using a computer, such as a microcomputer or a central processing unit (CPU), and hardware, such as a peripheral circuit of the computer or a peripheral device. The path plan generation device 20 includes an information input unit 21, an information storage unit 22, a path plan generation unit 23, and an information output unit 24, as a functional configuration configured of a combination of the hardware and software such as a program executed by the computer.

The information input unit 21 receives, from the outside, inputs of topographic shape information indicating the topographic shape of the ground surface or the like of the ground G in the construction target area A, a position of the work vehicle 100, and design surface information indicating a target shape in the construction target area A. The design surface information is acquired from, for example, an external computer aided design (CAD) system that designs the construction site including the construction target area A. The topographic shape information is acquired by a detection device, such as a radar, provided in the work vehicle 100. The detection device may be provided in another work vehicle, or may be attached to a structure in the construction area A. In addition, the detection device may be mounted on a flying object that flies over the construction site. An example of the flying object includes an unmanned aerial vehicle (UAV) such as a drone.

The information storage unit 22 stores the topographic shape information and the design surface information, which are received through the information input unit 21. In addition, the information storage unit 22 stores various types of vehicle information related to the work vehicle 100, for example, the size of the blade 133, the travel driving force by the crawlers 110a and 110b, and the travel maximum speed.

As will be described in detail below, the path plan generation unit 23 generates a path plan for performing an excavation construction in the construction target area A by the work vehicle 100.

The information output unit 24 outputs information about the path plan generated by the path plan generation unit 23 to the outside.

(Configuration Example of Path Plan Generation Unit 23)

FIG. 4 is a diagram showing an example of a movement path L of the work equipment 130 in the travel path R of the work vehicle 100 according to the embodiment.

The path plan generation unit 23 generates a work equipment path plan and a travel path plan of the work vehicle 100, as path plans for performing the excavation construction in the construction target area A by the work vehicle 100. The movement path L of the work equipment 130 indicates, in a case where the work vehicle 100 travels along the travel path R, positions and angles of the blade 133 in a vertical direction at a plurality of positions on the travel path R.

The path plan generation unit 23 generates the work equipment path plan that is an optimal movement path L of the work equipment 130, and the travel path plan that is an optimal travel path R of the work vehicle 100. The path plan generation unit 23 generates the work equipment path plan and the travel path plan in which slip of the crawlers 110a and 110b of the work vehicle 100 does not occur and the excavation efficiency by the blade 133 is improved. In the path plan generation unit 23, reinforcement learning is performed to generate the work equipment path plan that is the optimal movement path L of the work equipment 130, and the travel path plan that is the optimal travel path R of the optimal work vehicle 100, for a plurality of candidates for the movement path L and the travel path R. In the present embodiment, a plurality of times of simulations for the plurality of candidates for the movement path L and the travel path R in which parameters related to an operation of the work equipment 130 are variously changed are performed while performing learning by the reinforcement learning.

FIG. 5 is a diagram showing an example of a simulation model used for a simulation for candidates for the movement path L and the travel path R in the path plan generation unit 23 according to the embodiment.

The path plan generation unit 23 decomposes a phenomenon that occurs during the excavation by the work vehicle 100 into a model for each element, in order to perform the simulation on the candidates for the movement path L and the travel path R. For example, as shown in FIG. 5, in the path plan generation unit 23, the phenomena occurring during the excavation by the work vehicle 100 is decomposed into a vehicle body model M10, a control model M20, and an earth model M30, and the reinforcement learning is performed by the simulation. The simulation is executed by a simulator that is operated on a computer to execute a simulation.

In the vehicle body model M10, parameters related to the operation of the work vehicle 100 on the travel path R and parameters related to the operation of the work equipment 130 on the travel path R are related. More specifically, the vehicle body model M10 includes, for example, a hydraulic pressure model M11 related to hydraulic equipment, such as a lift cylinder and a tilt cylinder, of the work equipment 130, a mechanism model M12 related to a mechanism of a movable part, such as the lift frame 131 and the blade 133, and a vehicle-underbody model M13 related to the undercarriage 110.

In the hydraulic pressure model M11, for example, a relief pressure or the like related to the hydraulic equipment, such as the lift cylinder and the tilt cylinder, of the work equipment 130 is simulated. In such hydraulic equipment, a maximum value of a reaction force caused by the operation of the lift frame 131 and the blade 133 is set as a maximum reaction force restriction. In the hydraulic pressure model M11, the relief pressure or the like of the hydraulic equipment, which is caused by the operation of the work equipment 130, is simulated within a range not exceeding the maximum reaction force restriction.

In the mechanism model M12, the simulation is performed for an excavation range in which the blade 133 can excavate in a case where the movable part, such as the lift frame 131 and the blade 133, is operated based on the movement path L. A position restriction of a tip blade is set in the blade 133. In the mechanism model M12, the simulation of the excavation operation by the blade 133 is performed within a range not exceeding the position restriction of the tip blade.

In the vehicle-underbody model M13, the simulation is performed for a vehicle speed due to the drive of the undercarriage 110, a degree of occurrence of the slip of the crawlers 110a and 110b, and the like. The vehicle speed in a case where the work vehicle 100 travels and moves along the travel path R by the drive of the undercarriage 110 is calculated based on the travel driving force (traction force) of the crawlers 110a and 110b and a reaction force received by pushing the earth of the ground G with the blade 133. For example, the degree of occurrence of the slip of the crawlers 110a and 110b can be simulated based on the reaction force received from the earth in a case where the lift frame 131 and the blade 133 perform the excavation operation. This reaction force is a sum of an excavation resistance (shear resistance) in a case where the blade 133 excavates the ground G and an earth moving resistance due to friction in a case where the earth held in front of the blade 133 is pushed to the front. In the vehicle-underbody model M13, in a case where the reaction force reaches the maximum reaction force restriction, determination can be made that a shoe slip limit of the crawlers 110a and 110b is exceeded and the slip occurs.

The control model M20 relates to a control condition in a case where the simulation is performed based on the candidates for the movement path L and the travel path R. The control model M20 has, for example, a path tracking model M21 and a path plan model M22.

In the path tracking model M21, it is assumed that, in a case where the simulation is performed, trajectory trackability to the travel path R of the work vehicle 100 and trajectory trackability to the movement path L of the work equipment 130 each have trackability of 100%, for example.

In addition, in the path plan model M22, the path plan model M22 for the plurality of movement paths L and the travel paths R, which are learning targets in performing the plurality of times of simulations, is set. In the path plan model M22, new movement paths L and travel paths R are generated such that the excavation efficiency is further improved, based on results of the simulations performed for the candidates for the movement path L and the travel path R. As shown in FIG. 4, in the path plan model M22, for example, a plurality of candidates for the movement path L to be simulated are sequentially generated by variously changing conditions, such as a start point P1 of the movement path L, an excavation start position P2 at which the blade 133 is lowered to start the excavation of the ground G, an excavation depth P3 at which the ground G is excavated by one excavation, an entry angle P4 of the blade 133 with respect to the ground G, an end point P5 of the movement path L, and the vehicle speed of the work vehicle 100, which are provisionally set along the travel path R. In the path plan model M22, the simulation is performed for one candidate for the movement path L, then another movement path L having a different condition is generated, and the simulation is sequentially executed.

The earth model M30 is for simulating parameters related to the earth of the ground G to be excavated by the work equipment 130 in a case where the work vehicle 100 is caused to travel based on each candidate of the movement path L and the travel path R to move the work equipment 130. The earth model M30 is classified into a topographic model M31 related to a topographic change caused by the excavation of the ground G with the blade 133 and a reaction force model M32 related to the reaction force received by the blade 133 in a case where the ground G is excavated.

In the topographic model M31, the simulation is performed for an excavation earth amount to be excavated by the excavation of the ground G with the blade 133. The excavation earth amount is calculated based on a difference between the topographic shape information of the ground G before the excavation and the design surface information for the design surface S, in a region along the travel path R. In the topographic model M31, in a case where the excavation from the ground G to the design surface S is divided into a plurality of times, the excavation earth amount at each time of the excavation can be calculated based on a difference between the topographic shape before the excavation and an excavation surface formed by one excavation along the movement path L.

FIG. 6 is a diagram showing another example of the movement path L of the work equipment 130 in the travel path R of the work vehicle 100 according to the embodiment.

In addition, as shown in FIG. 6, in a case where the ground G is excavated by the blade 133 based on the movement path L and an earth D1 is held in front of the blade 133, the simulation is performed for a holding earth amount, which is the amount of the held earth D1, in the topographic model M31.

FIG. 7 is a diagram showing still another example of the movement path L of the work equipment 130 in the travel path R of the work vehicle 100 according to the embodiment.

As shown in FIG. 7, in a case where the other part of the ground G on the travel path R is backfilled by an earth D2 obtained by the excavation of a part of the ground G on the travel path R with the blade 133, the simulation may be performed for an actual excavation earth amount based on a difference between the excavation earth amount and a backfill earth amount, which is an amount of the backfilled earth D2, in the topographic model M31.

FIG. 8 is a side view of the earth D1 held in front of the blade 133 and an earth D5 of windrows formed by overflowing onto both sides of the blade 133 in a case where the excavation is performed by the work equipment 130 according to the embodiment. FIG. 9 is a plan view of the earth D1 held in front of the blade 133 and the earth D5 of the windrows formed by overflowing onto both sides of the blade 133 in a case where the excavation is performed by the work equipment 130 according to the embodiment.

In the topographic model M31, as shown in FIGS. 8 and 9, in a case where the ground G is excavated by the blade 133 based on the movement path L and the travel path R, the simulation is performed for an amount of the earth D5 of the so-called windrows that overflow onto both sides of the blade 133 in a width direction. In the topographic model M31, in a case where the ground G is excavated by the blade 133 based on the movement path L along one travel path R to form the windrow on the ground G, the amount of the earth D5 of the formed windrow is included in the topographic information in a case where the simulation is performed for the candidate for the movement path L in another travel path R.

The amount of the held earth D1 and the amount of the earth D5 of the windrow may be calculated, for example, based on a preliminary experiment using a model.

In addition to the above, in the topographic model M31, for example, in a case where a part of the earth excavated on the travel path R is backfilled at another position on the travel path R, the simulation is performed by assuming that the backfilled earth is rolled with the crawlers 110a and 110b for compaction.

In the topographic model M31, the simulation is also performed for a landslide for the earth excavated on the travel path R.

In the reaction force model M32, in a case where the blade 133 is operated while causing the work vehicle 100 to move along the travel path R to excavate the ground G based on the movement path L, the simulation is performed for the earth moving resistance that occurs between the blade 133 and the ground G in a case where the earth D1 held in front of the blade 133 is pushed to the front. In a case where the work vehicle 100 moves forward along the travel path R, the amount of the earth D1 held in front of the blade 133 gradually increases. That is, the earth moving resistance changes every moment.

In the reaction force model M32, in a case where the blade 133 is operated along the movement path L, the simulation is performed for the excavation resistance received by the blade 133 from the earth of the ground G. The excavation resistance increases with the excavation depth by the blade 1330 into the ground G. As shown in FIG. 7, in a case where the earth D2 is backfilled in front of the end point P5 of the movement path L, the reaction force model M32 also simulates the excavation resistance received by the blade 133 from the backfilled earth D2.

The simulator performs the reinforcement learning on the path plan generation unit 23 based on the results of the plurality of times of simulations performed for the plurality of candidates for the movement path L and the travel path R. For each simulation, based on the excavation earth amount in a case where the work equipment 130 is operated along the movement path L, while causing the work vehicle 100 to move along the travel path R, and a work time required for the work equipment 130 to perform the excavation from the start point to the end point of the movement path L, the simulator calculates a reward by the reinforcement learning, for example, based on the following equation (1).

Reward = excavation ⁢ earth ⁢ amount / work ⁢ time ( 1 )

In addition, for each simulation, in a case where the work equipment 130 is operated along the movement path L while causing the work vehicle 100 to move along the travel path R, the simulator provides a penalty value (for example, −0.05) in the reinforcement learning in a case where the reaction force received by the blade 133 from the earth exceeds the shoe slip limit of the undercarriage 110.

With the reinforcement learning, the simulator searches for optimal movement path L and travel path R, based on the calculated reward and the penalty value, and the path plan generation unit 23 learns to generate the work equipment path plan and the travel path plan.

The path plan generation unit 23 is learned to generate the work equipment path plan and the travel path plan for the entire construction target area A based on the work equipment path plan, which is the optimum movement path L, and the travel path plan, which is the optimum travel path R.

FIG. 10 is a diagram in which the earth D1 held in front of the blade 133 and the earth D5 of the windrows formed by overflowing onto both sides of the blade 133, in a case where the excavation is performed by the work equipment 130 according to the embodiment, are modeled as polygonal-shaped three-dimensional models Dm1 and Dm5.

Meanwhile, as shown in FIGS. 8 to 10, in the topographic model M31 of the path plan generation unit 23 as described above, the earth D1 held in front of the blade 133 is simplified by modeling the earth D1 as the polygonal prism-shaped (for example, triangular prism-shaped) three-dimensional model Dm1 to calculate the holding earth amount. Accordingly, the holding earth amount that changes every moment in accordance with the excavation of the ground G based on the movement path L can be efficiently calculated.

In addition, in the topographic model M31, the amount of the earth D5 of the windrows that protrude on both sides of the blade 133 is calculated in a simplified manner by modeling the earth D5 as the polygonal prism-shaped (for example, quadrangular prism-shaped (rectangular prism)) three-dimensional model Dm5 in accordance with a ratio of the earth D1 held in the front. Accordingly, the amount of the earth D5 of the windrows that changes every moment in accordance with the excavation of the ground G based on the movement path L can be efficiently calculated.

(Operation Example of Path Plan Generation Device 20)

FIG. 11 is a flowchart showing an operation of the path plan generation device 20 according to the embodiment.

The path plan generation device 20 learned by the reinforcement learning is provided in, for example, the work vehicle 100. The path plan generation device 20 generates the path plan for performing the excavation work on the ground G of the construction target area A by the work vehicle 100.

For this purpose, as shown in FIG. 11, first, the information input unit 21 acquires, from the outside, the topographic shape information indicating the shape of the ground G in the construction target area A, the position of the work vehicle 100, and the design surface information indicating a shape required to be excavated of the ground G in the construction target area A (S1). Next, the information storage unit 22 stores the acquired topographic shape information, position of the work vehicle 100, and design surface information (S2). Next, the path plan generation unit 23 generates the work equipment path plan indicating the movement path L of the work equipment 130 and the travel path plan indicating the travel path R of the work vehicle 100, based on the topographic shape information, the position of the work vehicle 100, and the design surface information (S3). Thereafter, the information output unit 24 outputs the generated work equipment path plan and travel path plan to the outside (S4).

FIG. 12 is a diagram showing an outline of a flow in which the path plan generation unit 23 is learned through the reinforcement learning according to the embodiment. The reinforcement learning can be performed, for example, by using the simulator that operates on a computer outside the vehicle body 100 to execute the simulation.

For this purpose, first, the simulator sets the candidate for the movement path L of the work equipment 130 and the candidate for the travel path R of the work vehicle 100 (S31).

The simulator performs the simulation based on the set candidates for the movement path L and the travel path R of the work vehicle 100. Specifically, the simulation for the operations of the work vehicle 100 and the work equipment 130 is performed using the vehicle body model M10 and the control model M20 (S32). In the simulation of the operation of the work vehicle 100, a travel trajectory of the work vehicle 100 along the travel path R in a case where the work equipment 130 is operated is simulated based on the set candidates for the movement path L and the travel path R of the work vehicle 100. In addition, in the simulation of the operation of the work equipment 130, a movement trajectory of the work equipment 130 in a case where the work equipment 130 is operated based on the movement path L is simulated.

In addition, based on data of the calculated travel trajectory of the work vehicle 100 and movement trajectory of the work equipment 130, the simulator calculates the work time in a case where the excavation is performed based on the movement path L and the travel path R.

Further, based on the data of the travel trajectory of the work vehicle 100 and the movement trajectory of the work equipment 130, the simulator executes the simulation for the earth in a case where the earth of the ground G is excavated, by using the earth model M30 (S33). In the earth model M30, for example, the excavation resistance in a case where the earth of the ground G is excavated is calculated. Data of the calculated excavation resistance is fed back to the vehicle body model M10 and is reflected in the calculation of the vehicle speed or the like of the work vehicle 100.

In addition, the simulator uses the earth model M30 to calculate a post-excavation topography indicating the shape of the ground G after the excavation by the work equipment 130. Further, the path plan generation unit 23 calculates the excavation earth amount by the work equipment 130, the holding earth amount, the earth amount of the windrow, and the like, using the earth model M30.

The simulator learns the path plan generation unit 23 by the reinforcement learning based on the work time and the excavation earth amount calculated as described above (S34). In the reinforcement learning, the reward and the penalty are calculated to evaluate the candidates for the movement path L and the travel path R. In this case, the simulator determines whether or not the excavation resistance exceeds the shoe slip limit of the crawlers 110a and 110b in a case where the work equipment 130 performs the excavation based on the movement path L while causing the work vehicle 100 to move along the travel path R. As a result, in a case where the excavation resistance exceeds the shoe slip limit, determination is made that the slip of the work vehicle 100 occurs.

In the simulator, after the simulation for one candidate for the movement path L and the travel path R is performed, the learning based on an evaluation result for the candidate for the movement path L and the travel path R is performed, and the candidate for the movement path L or the travel path R required to be simulated next is set. The candidate for the movement path L or the travel path R required to be simulated next is set such that the slip of the work vehicle 100 does not occur and the reward represented by the above equation (1) is as large as possible.

The simulation for the candidate for the movement path L and the travel path R is repeated a plurality of times while performing the learning by the reinforcement learning. Accordingly, the path plan generation unit 23 is learned such that the slip of the work vehicle 100 does not occur and the work equipment path plan and the travel path plan having a high excavation efficiency can be generated.

(Configuration of Work Vehicle Path Plan Generation System)

FIG. 13 is a schematic block diagram showing a configuration example of a work vehicle path plan generation system 50 according to the embodiment.

As shown in FIG. 13, the work vehicle path plan generation system 50 includes a remote control device 60 and the work vehicle 100.

The remote control device 60 includes a communication unit 61, an information output unit 62, and a supervision unit 63.

The communication unit 61 can communicate with the work vehicle 100 by a public wireless communication network and a wireless communication unit.

The information output unit 62 outputs information necessary for automatically driving the work vehicle 100. The information output unit 62 acquires, for example, the design surface information or the like of the construction target area A from an external CAD system or the like, and transmits the acquired information to the work vehicle 100 via the communication unit 61.

The supervision unit 63 monitors an operation state of each unit of the work vehicle 100 based on information detected by various sensors provided in the work vehicle 100.

The work vehicle 100 is automatically operated based on the path plan generated by the path plan generation device 20.

Therefore, the work vehicle 100 includes a communication unit 71, the path plan generation device 20, a position detection unit 72, a path plan storage unit 73, and a vehicle control unit 74.

The communication unit 71 can communicate with the communication unit 61 of the remote-control device 60 by a public wireless communication network and a wireless communication unit.

A part or all of the configuration of the work vehicle 100 may be provided in the remote control device 60. For example, the path plan generation device 20 described above is provided on the work vehicle 100 side in the present embodiment. The path plan generation device 20 may be provided on the remote control device 60 side. In addition, a part or all of the configuration of the remote control device 60 may be provided in the work vehicle 100.

The position detection unit 72 is provided in the work vehicle 100. The position detection unit 72 can detect the position of the work vehicle 100 using, for example, a GPS or the like.

The path plan storage unit 73 stores the work equipment path plan and the travel path plan of the work vehicle 100, which are generated by the path plan generation device 20 and are output to the outside.

The vehicle control unit 74 controls the operation of each unit of the work equipment 130 and the work vehicle 100, based on the work equipment path plan and the travel path plan stored in the path plan storage unit 73. The work vehicle 100 operates the work equipment 130 while causing the work vehicle 100 to move based on the work equipment path plan, which is the optimum movement path L of the work equipment 130, and the travel path plan, which is the optimum travel path R of the work vehicle 100. Accordingly, the excavation of the ground G is efficiently performed.

Actions and Effects

According to the present embodiment, the path plan of the work vehicle 10 can be efficiently generated.

Although the embodiments of the present disclosure have been described with reference to the drawings, the specific configuration is not limited to the above-described embodiments, and the design change and the like within a range not departing from the scope of the present disclosure are also included.

For example, the work vehicle 100 according to the embodiments described above is a bulldozer, but the present disclosure is not limited thereto. For example, the work vehicle 100 may be a work machine having work equipment, such as a hydraulic excavator, a wheel roller, a motor grader, and an undercarriage.

In addition, a part or all of a program executed by a computer in the embodiments described above can be distributed via a computer-readable recording medium or a communication line.

INDUSTRIAL APPLICABILITY

According to one aspect described above, the path plan of the work vehicle can be efficiently generated.

REFERENCE SIGNS LIST

10 . . . work vehicle 20 . . . path plan generation device 22 . . . information storage unit
23 . . . path plan generation unit 50 . . . work vehicle path plan generation system
72 . . . position detection unit 73 . . . path plan storage unit 74 . . . vehicle control unit
100 . . . work vehicle 130 . . . work equipment A . . . construction target area D1 . . . earth
D5 . . . earth Dm1, Dm5 . . . three-dimensional model G . . . ground L . . . movement path
R . . . travel path S . . . design surface