ELECTRONIC DEVICE, AND INFORMATION PROCESSING SYSTEM

Description

TECHNICAL FIELD

The present disclosure relates to an electronic device and an information processing technology using the electronic device.

BACKGROUND

Disclosed in PTL 1 is an information processing system having an information processing apparatus, a trolley, and a mat. The trolley is equipped with an image sensor (camera) that captures an image of an array pattern printed on the mat, and configured to acquire location information and control information on the mat by reading the array pattern. The information processing apparatus acquires the location information and control information from the trolley, and controls the operation of the trolley. The trolley is equipped with a battery that can be recharged by an external power source, and is driven by power supplied from the battery.

CITATION LIST

Patent Literature

PTL 1

• • PCT Patent Publication No. WO2018/25467

SUMMARY

Technical Problem

When an electronic device using a battery runs out of battery power, the battery needs to be replaced with another battery or recharged. Therefore, if no battery for replacement is available or if neither a charger nor an external power source is available, power for the electronic device cannot be obtained. This results in the inability to operate the electronic device.

The present disclosure has been made in view of the above circumstances. An object of the present disclosure is to provide a technology related to electronic devices having neither primary batteries (disposable batteries) nor secondary batteries to be rechargeable by an external power source.

Solution to Problem

In order to solve the above problem, according to an aspect of the present invention, there is provided an electronic device including a power generation section that generates power, a power storage section that stores the power generated by the power generation section, an image sensor that captures an image of a periphery of the electronic device, and a display section that displays an image corresponding to a subject imaged by the image sensor. The image sensor and the display section are driven by the power stored in the power storage section.

According to another aspect of the present invention, there is provided an information processing system including an electronic device, and an information processing apparatus that has a camera which captures an image of the electronic device. The electronic device includes a power generation section that generates power, a power storage section that stores the power generated by the power generation section, and a first display section that is driven by the power stored in the power storage section, to display an image. The information processing apparatus includes a recognition section that recognizes the image captured by the camera, and a second display section that displays an image according to a result of recognition by the recognition section.

Any combinations of the above-mentioned component elements, and any conversions of expressions of the present invention between, for example, methods, apparatuses, systems, computer programs, recording media storing readable computer programs, and data structures are also effective as the aspects of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a state where an electronic device according to an embodiment is disposed on a mat.

FIG. 2 is a diagram illustrating a front view of the electronic device.

FIG. 3 is a diagram illustrating a side view of the electronic device.

FIG. 4 is a diagram illustrating a bottom view of the electronic device.

FIG. 5 is a diagram illustrating the functional blocks of the electronic device.

FIG. 6 is a flowchart illustrating the operations of component elements of the electronic device.

FIG. 7 is a diagram illustrating a state where an object image is displayed on a display section.

FIG. 8 is a diagram illustrating an example of an information processing system that includes the electronic device and an information processing apparatus.

FIG. 9 is a diagram illustrating the functional blocks of the information processing apparatus.

DETAILED DESCRIPTION

FIG. 1 illustrates a state where an electronic device 10 according to an embodiment is disposed on a mat 2. The electronic device 10 according to the embodiment may be a toy that a user can hold in his/her hand and play with. The mat 2 is partitioned into a plurality of grid-like regions, and a code is printed or attached to each region. In the embodiment, the code may include at least location information, which indicates the location of the region of the mat 2, and information regarding an object associated with the region (hereinafter referred to as “object information”). The code may be a one-dimensional code or a two-dimensional code, or may be human-readable text as indicated in FIG. 1. In any case, the code includes information that is readable by the electronic device 10.

The electronic device 10 includes a housing 20 having a polyhedral shape. The housing 20 according to the embodiment preferably has such a size as to allow the user to hold the housing 20 with one hand of the user. The housing 20 may be shaped like a rectangular parallelepiped that is approximately 2 to 3 cm high and has approximately 3 to 5 cm long horizontal and vertical sides. The horizontal and vertical sides may have the same length. The electronic device 10 includes a display section 12 on its top surface, and an image sensor on its bottom surface. The display section 12 is rectangular in shape. The image sensor captures the image of a code attached to each region of the mat 2. The embodiment is configured such that, when the image sensor captures the image of the code, the display section 12 displays an image corresponding to the object information included in the captured code. The displayed image may be an image representing the object.

The electronic device 10 according to the embodiment may be capable of traveling. The electronic device 10 travels on the mat 2 and stops within a partitioned region. For example, in a case where a code included in each area includes destination information indicating the location of the next region to be reached, the electronic device 10 may read the destination information included in the code to autonomously travel to the next region to be reached. It should be noted that the electronic device 10 may travel on the mat 2 according to a travel instruction from an external control apparatus (not depicted).

FIG. 2 illustrates a front view of the electronic device 10, FIG. 3 illustrates a side view of the electronic device 10, and FIG. 4 illustrates a bottom view of the electronic device 10. The electronic device 10 according to the embodiment has two wheels 14a and 14b (hereinafter referred to as “wheels 14” unless specifically distinguished) on the bottom surface of the housing 20. Each wheel 14 is coupled to an electric motor.

The image sensor 16 is a camera for capturing an image of the periphery of the electronic device 10, and is disposed on the bottom surface of the housing 20. Therefore, the image sensor 16 captures an image below the housing 20. Note that the image sensor 16 may alternatively be disposed, for example, on the front, the right side, the rear, the left side, or the top surface of the housing 20. The electronic device 10 according to the embodiment has a function of analyzing the image captured by the image sensor 16 to identify a subject, and displaying an image corresponding to the subject on the display section 12. More specifically, the electronic device 10 identifies a code included in the captured image, and causes the display section 12 to display an image corresponding to the object information included in the code.

A lighting section 18 has an LED that illuminates, and is disposed on the bottom surface of the housing 20. The lighting section 18 protrudes downward from the bottom surface of the housing 20. When the electronic device 10 is disposed on the mat 2 with its top surface facing up (its bottom surface facing down), three points, namely, the wheels 14a and 14b and the lighting section 18, are in contact with the ground. The electronic device 10 is able to travel on the mat 2 by rotating the wheels 14a and 14b.

FIG. 5 illustrates the functional blocks of the electronic device 10. The electronic device 10 includes the display section 12, the wheels 14, the image sensor 16, the lighting section 18, a power generation section 30, a power storage section 32, an inertial measurement unit (hereinafter referred to as an “IMU”) 34, a control section 36, and a communication section 42. The IMU 34 may be an attitude sensor that includes a three-axis acceleration sensor and a three-axis gyro sensor. The control section 36 includes a first control section 38 and a second control section 40.

The electronic device 10 includes a computer, and various functions are implemented when the computer executes programs. The computer includes hardware such as a memory into which the programs are loaded, one or more processors for executing the loaded programs, an auxiliary storage, and other LSIs. Each of the processors includes a plurality of electronic circuits such as semiconductor integrated circuits and LSIs. The plurality of electronic circuits may be mounted on a single chip or on multiple chips. The functional blocks depicted in FIG. 5 are implemented by cooperation between hardware and software. Therefore, it will be understood by those skilled in the art that these functional blocks can be implemented in various forms by hardware only, by software only, or by a combination of hardware and software.

The electronic device 10 according to the embodiment is not equipped with a primary battery or a secondary battery rechargeable by an external power source such as a commercial power source. The electronic device 10 according to the embodiment is configured such that the power generation section 30 has a power generation function and generates power that is used by the electronic device 10. The power storage section 32 stores the power generated by the power generation section 30, and the power stored in the power storage section 32 drives the display section 12, the image sensor 16, the lighting section 18, the IMU 34, the control section 36, and the communication section 42.

FIG. 6 is a flowchart illustrating examples of operations performed by individual component elements of the electronic device 10. In the embodiment, the power generation section 30 includes electric motors that generate power according to the rotation of the wheels 14. When the user holds the electronic device 10 in his/her hand and rotates the wheels 14, the electric motors generate the power (step S10). When the user rotates the wheels 14 by moving the electronic device 10 forward and backward with the wheels 14 placed in contact with a floor or table, the electric motors coupled to the wheels 14 generate the power. The power storage section 32 includes, for example, a capacitor, and stores the generated power (step S12).

When power is stored in the power storage section 32, the first control section 38 immediately becomes activated (step S14), and monitors the amount of power stored in the power storage section 32 (step S16). The first control section 38, which is a microcontroller, controls the activation of the second control section 40 according to the monitored amount of power stored. The first control section 38 may monitor the amount of power stored in the power storage section 32, by monitoring the voltage of the power storage section 32.

When the amount of power stored in the power storage section 32 is smaller than a predetermined value (“N” in step S16), the first control section 38 waits until the user further rotates the wheels 14 to increase the amount of power stored. When the amount of power stored in the power storage section 32 is equal to or greater than the predetermined value (“Y” in step S16), the first control section 38 illuminates the lighting section 18 for a predetermined period of time (e.g., 0.5 seconds) (step S18), supplies power from the power storage section 32 to the second control section 40, and activates the second control section 40 (step S20). Preferably, the first control section 38 supplies the power stored in the power storage section 32 to the second control section 40 on condition that the amount of power stored in the power storage section 32 becomes a predetermined amount of power stored with which the second control section 40 can steadily operate. By confirming that the lighting section 18 is illuminated, the user recognizes that sufficient power is stored in the electronic device 10. Note that lighting control of the lighting section 18 may be performed by the activated second control section 40.

Upon receiving the power supplied from the power storage section 32, the second control section 40 is able to control the drive of the display section 12, the image sensor 16, the IMU 34, and the communication section 42. Note that, immediately upon activation, the second control section 40 may operate in a low power consumption mode in order to reduce power consumption. In the low power consumption mode, the second control section 40 may reduce the power consumption by not driving at least the image sensor 16. Further, the second control section 40 may reduce the power consumption by not driving the display section 12 and the communication section 42 in the low power consumption mode.

In the low power consumption mode, the second control section 40 supplies power to the IMU 34, and the IMU 34 provides the second control section 40 with measurement results at predetermined intervals. According to the measurement results provided from the IMU 34, the second control section 40 determines whether the electronic device 10 is stationary or not (step S22). If the electronic device 10 is not stationary (“N” in step S22), the user may be rotating the wheels 14 of the electronic device 10 in order to cause the power generation section 30 to generate power, or may be moving the electronic device 10 to dispose it in any region of the mat 2. In either case, when the electronic device 10 is not stationary, it is unlikely that the electronic device 10 is in a state where it is able to capture an image of the code attached to the mat 2. In such a situation, therefore, the second control section 40 avoids unnecessary power consumption by maintaining a state where no power is supplied to the image sensor 16 so as not to drive the image sensor 16 (step S24).

Meanwhile, if the electronic device 10 is stationary (“Y” in step S22), there is a possibility that the electronic device 10 is disposed on the mat 2 and able to capture an image of the code attached to the mat 2. Therefore, the second control section 40 supplies power to the image sensor 16 to drive it (step S26). The image sensor 16 periodically captures an image of the periphery of the electronic device 10, and the second control section 40 analyzes the image captured by the image sensor 16 to search whether the captured image includes the code attached to the mat 2 (step S28). Upon reading a new code (“Y” in step S28), the second control section 40 identifies the object information contained in the code, and causes the display section 12 to display an object image corresponding to the object information (step S30). Note that, when the read code is new, it indicates that the code read this time is different from the code read last time.

FIG. 7 illustrates a state where an object image is displayed on the display section 12. The second control section 40 identifies the object information from the read code, and causes the display section 12 to display an object image corresponding to the object information. The electronic device 10 retains a database (not depicted) in which object images corresponding to the object information are recorded, and the second control section 40 reads the object image from the database and displays the read object image on the display section 12.

As described above, the code is provided for each of the regions into which the mat 2 is partitioned. Therefore, each time the electronic device 10 is disposed in a region different from the previous one, the second control section 40 causes the display section 12 to display an object image associated with the region where the electronic device 10 is disposed. The display section 12 may include electronic paper having image storage properties. After displaying an image once, the electronic paper continues to display the image without consuming power, and thus contributes to power saving in the electronic device 10.

In step S28, if the code read by the second control section 40 is the same as the previous code (“N” in step S28), the second control section 40 does not update the display on the display section 12. Therefore, in a case where the electronic device 10 continues to be disposed in the same region, the display section 12 continues to display the same object image. If the amount of power stored is not zero (“N” in step S32), the second control section 40 repeatedly performs steps S22 and beyond.

Meanwhile, if the amount of power stored is zero (“Y” in step S32), the processing depicted in the flowchart of FIG. 6 terminates.

According to the amount of power stored in the power storage section 32, the second control section 40 may control imaging intervals at which the image sensor 16 captures an image. The second control section 40 may set relatively long imaging intervals for the image sensor 16 when the amount of power stored is small, and may set relatively short imaging intervals for the image sensor 16 when the amount of power stored is large.

Further, according to the amount of power stored in the power storage section 32, the second control section 40 may control the contents to be displayed on the display section 12. In a case where a small amount of power is stored, the second control section 40 may cause the display section 12 to display the contents that consume a relatively small amount of power. Meanwhile, in a case where a large amount of power is stored, the second control section 40 may cause the display section 12 to display the contents that consume a relatively large amount of power. More specifically, when a relatively large amount of power is stored, the second control section 40 may display images with rich colors or animations and increase a frequency with which an image update is performed. For example, in a case where the above-mentioned database is configured such that single object information corresponds to a plurality of object images, the second control section 40 may determine, according to the amount of power stored, a frequency with which the object image is changed.

Note that it is preferable that the second control section 40 erase an image displayed on the display section 12 when the amount of power stored in the power storage section 32 falls below a predetermined threshold. Since electronic paper is used as the display section 12 in the embodiment, the display section 12 continues to display the object image even after the amount of power stored is reduced to zero. Therefore, it is preferable that the second control section 40 prevent the object image from being continuously displayed, by erasing the image displayed on the display section 12 immediately before the amount of power stored is reduced to zero.

In the electronic device 10, the electric motor included in the power generation section 30 is driven by the power stored in the power storage section 32, to rotate the wheels 14 while no power is generated by the power generation section 30. The second control section 40, which is capable of controlling the rotation of the electric motor while no power is generated, may provide autonomous traveling on the mat 2 according to the destination information included in the code. The space of the housing 20 can effectively be utilized by using the electric motor for both power generation and traveling in the above-described manner.

FIG. 8 illustrates an example of an information processing system 1 that includes the electronic device 10 and an information processing apparatus 50. The information processing apparatus 50 includes a camera that captures an image of the electronic device 10. The information processing apparatus 50, which is a smartphone, a tablet computer, or other mobile terminals or a smart glass, generates an augmented reality (AR) image and displays the generated AR image on a display section 58.

FIG. 9 illustrates the functional blocks of the information processing apparatus 50. The information processing apparatus 50 includes a camera 52, an image recognition section 54, a processing section 56, and the display section 58.

The information processing apparatus 50 includes a computer that implements various functions by executing the programs. The computer includes hardware such as a memory into which the programs are loaded, one or more processors for executing the loaded programs, an auxiliary storage device, and other LSIs. Each of the processors includes a plurality of electronic circuits such as semiconductor integrated circuits and LSIs. The plurality of electronic circuits may be mounted on a single chip or on multiple chips. The functional blocks depicted in FIG. 9 are implemented by cooperation between hardware and software. Therefore, it will be understood by those skilled in the art that these functional blocks can be implemented in various forms by hardware only, by software only, or by a combination of hardware and software.

The processing section 56 causes the display section 58 to display a camera image that is obtained when the camera 52 captures an image of a real space. In the example depicted in FIG. 8, the camera 52 captures an image of the electronic device 10, and the display section 58 displays the real space including the electronic device 10. Upon recognizing an object image included in the camera image, the image recognition section 54 provides the processing section 56 with the result of recognition, namely, information (object image ID) for identifying the recognized object image. The processing section 56 retains a database in which three-dimensional images corresponding to object image IDs are recorded, and reads a three-dimensional image from the database according to the object image ID identified by the image recognition section 54.

The processing section 56 superimposes the read three-dimensional image on the camera image displayed on the display section 58. As a result, a three-dimensional image according to the result of recognition by the image recognition section 54 is displayed in a manner being superimposed on the object image by the display section 58. In the above-described manner, the processing section 56 superimposes the three-dimensional image on the camera image, allowing the user to enjoy AR.

The present invention has been described above in conjunction with the embodiment. The embodiment is illustrative and not restrictive. It will be understood by those skilled in the art that the combination of individual component elements and the combination of individual processes can variously be modified and that such modifications are also within the scope of the present invention. The electronic device 10 may cause the communication section 42 to transmit information regarding the amount of charge to the information processing apparatus 50.

The embodiment has been described on the assumption that the power generation section 30 includes electric motors coupled to the wheels 14. However, the power generation section 30 may alternatively include a different type of power generation module. For example, the power generation section 30 may include a solar power generation module or a vibration power generation module. Further, the embodiment has been described on the assumption that the display section 12 includes electronic paper. However, the display section 12 may alternatively include a different type of display such as a liquid-crystal display or an organic EL display.

The information processing system 1 may include a plurality of electronic devices 10. The electronic devices 10 may cause the communication section 42 to inform each other of the amount of power stored, and the electronic device 10 having the largest amount of charge may cause the display section 12 to display information informing the user that the amount of charge is large. By knowing the information regarding the amount of charge, the user is able to obtain an opportunity to generate power. Note that the electronic devices 10 may have a non-contact power supply function, and the electronic device 10 having a large amount of charge may supply power to the electronic device 10 having a small amount of charge.

The embodiment has been described on the assumption that the electronic device 10 is a toy that the user can hold in his/her hand and play with. However, the electronic device 10 may not be a toy and may not be in such a size as to allow the user to hold in his/her hand. The electronic device 10 may be a device (apparatus) having the wheels 14 and capable of moving, such as a wheeled robot, a trolley, a vehicle, a walker, a wheelchair, a radio-controlled car, or a robotic vacuum cleaner, for example. Further, the electronic device 10 may be a device (apparatus) having no wheels 14, such as a smartphone, a tablet computer, or other portable terminal devices, furniture, or an electric appliance, for example.

The embodiment has been described on the assumption that the second control section 40 identifies the code included in the image captured by the image sensor 16 and causes the display section 12 to display an image corresponding to the code. Alternatively, however, the display section 12 may display an image corresponding to a subject other than the code. The image sensor 16 may be disposed on the front, the right side, the rear, the left side, or the top surface of the housing 20 in addition to the bottom surface of the housing 20 to capture an image of the environment around the housing 20, and the second control section 40 analyzes the captured image to identify the subject and displays an image corresponding to the subject on the display section 12. For example, the second control section 40 may identify a subject included in the captured image, such as an object, a person's face, a person's posture or movement, or an animal, and display an image corresponding to the subject on the display section 12.

Further, in order to increase the user's motivation for power generation, the information processing apparatus 50 in the information processing system 1 may cause the display section 58 to display, in AR, information for increasing the efficiency of power generation and information regarding the actual amount of power stored.

Note that the embodiment has been described on the assumption that the electronic device 10 is not equipped with a primary battery or a secondary battery rechargeable by an external power source. However, in a modification of the embodiment, a primary battery and/or a secondary battery rechargeable by an external power source may be mounted in the electronic device 10 as an auxiliary power supply. For example, the second control section 40 may use the power from the power storage section 32 as long as the power storage section 32 does not run out of the amount of power stored, and may use the auxiliary power supply when the power storage section 32 runs out of the amount of power stored.

INDUSTRIAL APPLICABILITY

The present disclosure is applicable to the technical field of electronic devices used by users.

REFERENCE SIGNS LIST

• • 1: Information processing system
  • 2: Mat
  • 10: Electronic device
  • 12: Display section
  • 14, 14a, 14b: Wheel
  • 16: Image sensor
  • 18: Lighting section
  • 20: Housing
  • 30: Power generation section
  • 32: Power storage section
  • 34: IMU
  • 36: Control section
  • 38: First control section
  • 40: Second control section
  • 42: Communication section
  • 50: Information processing apparatus
  • 52: Camera
  • 54: Image recognition section
  • 56: Processing section
  • 58: Display section