IMAGE-BASED LAUNDRY WASHING MACHINE LEVEL CALIBRATION

Description

BACKGROUND

Laundry washing machines are used in many single-family and multi-family residential applications to clean clothes and other fabric items. Due to the wide variety of items that may need to be cleaned by a laundry washing machine, many laundry washing machines provide a wide variety of user-configurable settings to control various aspects of a wash cycle such as water temperatures and/or amounts, agitation, soaking, rinsing, spinning, etc. The settings cycle can have an appreciable effect on washing performance, as well as on energy and/or water consumption, so it is generally desirable for the settings used by a laundry washing machine to appropriately match the needs of each load washed by the machine.

One particular area in which laundry washing machine performance may be sub-optimal is spinning a wash basket. It has been found that different spin speeds and/or durations are better suited for different types of loads, e.g., bedding, towels, cottons, delicates, athletic apparel, etc. Spinning at higher speeds generally removes more wash fluid, and does so more quickly, although doing so consumes more energy, generates greater noise, and can cause increased wear on clothing. In addition, bulky loads can often become unbalanced, such that higher speed spins may result in loud banging and vibrations, which can further lead to premature wear on a laundry washing machine. Lower speed spins, in contrast, are generally quieter and gentler on clothing, but are less effective, and may be insufficient for bulky and highly absorbent materials.

Further, while various control methodologies may be developed to optimize laundry washing machine performance, a significant challenge associated with such methodologies is the varied environments within which laundry washing machines may be installed, as a control methodology and/or the operational settings used thereby that are optimized for particular environmental conditions may not be optimal for installations that depart significantly from those environmental conditions. For example, installation of a laundry washing machine on a surface that is not level can cause excessive vibrations, particularly when spinning the load, and even when vibration-reducing structures such as suspension assemblies are used in the laundry washing machine. A wash basket that is not close to level can lead to excessive vibrations that can require a decreased spin speed, even when the load is not unbalanced. Where a load is unbalanced, these problems are exacerbated and can lead to even greater vibrations, increased noise, as well as wear on bearings and other kinetic components. These problems may also lead to increased out-of-balance events, which can increase the length of wash/spin operations, and in some instances, require a wash cycle to ultimately be restarted.

Therefore, a significant need also exists in the art for a manner of adapting the control methodologies and/or operational settings that may be used to optimize laundry washing machine performance for use in different installations.

SUMMARY

The invention addresses these and other problems associated with the art by providing a laundry washing machine and method that utilize one or more images captured of a wash tub to determine a level state of the wash tub. The level state of the wash tub determined in this manner may then be used to level the laundry washing machine and thereby optimize the performance thereof.

Therefore, consistent with one aspect of the invention, a laundry washing machine may include a wash tub suspended within a housing by a suspension assembly and configured to receive a load of laundry, one or more image sensors configured to capture image data of at least a portion of the wash tub, and a controller operably coupled to the one or more image sensors and configured to determine a level state of the wash tub using the captured image data output by the one or more image sensors.

In some embodiments, the controller is further configured to perform a wash cycle to wash the load of laundry in the wash tub. Also, in some embodiments, the suspension assembly is configured to allow for relative movement between the wash tub and the housing. Further, in some embodiments, the one or more image sensors includes a first image sensor positioned at a repeatable position and orientation with at least a portion of an interior of the wash tub in a field of view of the first image sensor. Some embodiments may further include an emitter configured to illuminate the wash tub while the one or more image sensors capture image data.

In some embodiments, the controller is further configured to determine a load type for the load of laundry using image data captured by the one or more image sensors. In addition, in some embodiments, the controller is further configured to determine a load color composition, a presence of excessive foam in the wash tub, a wash fluid level, a presence or absence of a removable agitator, a collapsed/extended state of a collapsible agitator, and/or a child presence using image data captured using the one or more image sensors. In some embodiments, the controller is configured to inhibit initiation of a wash cycle or dynamically determine a spin operation to be performed during the wash cycle based on the determined level state.

In addition, in some embodiments, the controller is configured to dispense a predetermined volume of water into the wash tub such that determining the level state of the wash tub is performed while the predetermined volume of water is in the wash tub. Moreover, in some embodiments, the controller is configured to determine the level state by locally performing image processing on the captured image data. In some embodiments, the controller is configured to determine the level state by communicating the captured image data to a remote device to perform image processing on the captured image data. Moreover, in some embodiments, the controller is configured to determine the level state by inputting the captured image data into a trained machine learning model.

In some embodiments, the controller is further configured to determine the level state of the wash tub by applying tare data determined during a calibration operation performed during manufacture of the laundry washing machine. In addition, in some embodiments, the tare data includes a calibration location of a registration points for the wash tub, and the controller is configured to determine the level state by determining a current location of a registration point for the wash tub and comparing the current location with the calibration location. In some embodiments, the controller is configured to determine the current location based on feature-based image registration of at least one visually distinct feature disposed on a wash basket or an agitation assembly housed within the wash tub. Moreover, in some embodiments, the controller is configured to determine the current location based on feature-based image registration of at least one dedicated registration indicia disposed within the wash tub.

Also, in some embodiments, the controller is further configured to, during a leveling operation, generate a level display on a user interface to guide a user as the user manually adjusts at least one height adjusters of the laundry washing machine. Some embodiments may also include at least one electromechanical height adjuster, and the controller is configured to actuate the at least one electromechanical height adjuster during a leveling operation in response to the determined level state.

Consistent with another aspect of the invention, a method of leveling a laundry washing machine of a type including a wash tub suspended within a housing by a suspension assembly and configured to receive a load of laundry may include, using one or more image sensors, capturing image data of at least a portion of the wash tub, and determining a level state of the wash tub using the captured image data output by the one or more image sensors.

Consistent with another aspect of the invention, a method of calibrating a laundry washing machine of a type including a wash tub suspended within a housing by a suspension assembly and configured to receive a load of laundry may include, using one or more image sensors, capturing image data of at least a portion of the wash tub while the laundry washing machine is disposed on a level surface, and determining tare data using the captured image data output by the one or more image sensors.

Other embodiments may include various methods of operating a laundry washing machine utilizing the various operations described above.

These and other advantages and features, which characterize the invention, are set forth in the claims annexed hereto and forming a further part hereof. However, for a better understanding of the invention, and of the advantages and objectives attained through its use, reference should be made to the Drawings, and to the accompanying descriptive matter, in which there is described example embodiments of the invention. This summary is merely provided to introduce a selection of concepts that are further described below in the detailed description, and is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in limiting the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a top-load laundry washing machine consistent with some embodiments of the invention.

FIG. 2 is a perspective view of a front-load laundry washing machine consistent with some embodiments of the invention.

FIG. 3 is a functional vertical section of the laundry washing machine of FIG. 1.

FIG. 4 is a block diagram of an example control system for the laundry washing machine of FIG. 1.

FIG. 5 is a top plan view of a wash tub of a laundry washing machine, illustrating various features suitable for image registration in connection with image-based level state determination consistent with some embodiments of the invention.

FIG. 6 is an example captured image of an impeller of a wash tub and suitable for image registration in connection with image-based level state determination consistent with some embodiments of the invention.

FIG. 7 is a flowchart illustrating an example operational sequence for calibrating the laundry washing machine of FIG. 1.

FIG. 8 is a flowchart illustrating an example operational sequence for leveling the laundry washing machine of FIG. 1.

FIG. 9 is an example display presented to a user during the operational sequence of FIG. 8.

FIG. 10 is a flowchart illustrating an operational sequence for implementing a wash cycle in the laundry washing machine of FIG. 1.

DETAILED DESCRIPTION

Embodiments consistent with the invention may be used in connection with leveling a laundry washing machine. In particular, in some embodiments consistent with the invention, one or more images captured of a wash tub, e.g., by using one or more image sensors of the laundry washing machine, may be used to determine a level state of the wash tub, thereby enabling the laundry washing machine to be leveled at an installation location to optimize the performance thereof.

Turning now to the drawings, wherein like numbers denote like parts throughout the several views, FIG. 1 illustrates an example laundry washing machine 10 in which the various technologies and techniques described herein may be implemented. Laundry washing machine 10 is a top-load washing machine, and as such includes a top-mounted door 12 in a cabinet or housing 14 that provides access to a vertically-oriented wash tub 16 housed within the cabinet or housing 14. Door 12 is generally hinged along a side or rear edge and is pivotable between the closed position illustrated in FIG. 1 and an opened position (not shown). When door 12 is in the opened position, clothes and other washable items may be inserted into and removed from wash tub 16 through an opening in the top of cabinet or housing 14. Control over washing machine 10 by a user is generally managed through a control panel 18 disposed on a backsplash and implementing a user interface for the washing machine, and it will be appreciated that in different washing machine designs, control panel 18 may include various types of input and/or output devices, including various knobs, buttons, lights, switches, textual and/or graphical displays, touch screens, etc. through which a user may configure one or more settings and start and stop a wash cycle.

The embodiments discussed hereinafter will focus on the implementation of the hereinafter-described techniques within a top-load residential laundry washing machine such as laundry washing machine 10, such as the type that may be used in single-family or multi-family dwellings, or in other similar applications. However, it will be appreciated that the herein-described techniques may also be used in connection with other types of laundry washing machines in some embodiments. For example, the herein-described techniques may be used in commercial applications in some embodiments. Moreover, the herein-described techniques may be used in connection with other laundry washing machine configurations. FIG. 2, for example, illustrates a front-load laundry washing machine 20 that includes a front-mounted door 22 in a cabinet or housing 24 that provides access to a horizontally-oriented wash tub 26 housed within the cabinet or housing 24, and that has a control panel 28 positioned towards the front of the machine rather than the rear of the machine as is typically the case with a top-load laundry washing machine. Implementation of the herein-described techniques within a front-load laundry washing machine would be well within the abilities of one of ordinary skill in the art having the benefit of the instant disclosure, so the invention is not limited to the top-load implementation discussed further herein.

FIG. 3 functionally illustrates a number of components in laundry washing machine 10 as is typical of many washing machine designs. For example, wash tub 16 may be vertically oriented, generally cylindrical in shape, opened to the top and capable of retaining water and/or wash liquor dispensed into the washing machine. Wash tub 16 may be supported by a suspension system such as a set of suspension rods 30 with corresponding vibration dampening springs 32.

Disposed within wash tub 16 is a wash basket 34 that is rotatable about a generally vertical axis A by a drive system 36. Wash basket 34 is generally perforated or otherwise provides fluid communication between an interior 38 of the wash basket 34 and a space 40 between wash basket 34 and wash tub 16. Drive system 36 may include, for example, an electric motor and a transmission and/or clutch for selectively rotating the wash basket 34. In some embodiments, drive system 36 may be a direct drive system, whereas in other embodiments, a belt or chain drive system may be used.

In addition, in some embodiments an agitator 42 such as an impeller, auger or other agitation element may be disposed in the interior 38 of wash basket 34 to agitate items within wash basket 34 during a washing operation. Agitator 42 may be driven by drive system 36, e.g., for rotation about the same axis as wash basket 34, and a transmission and/or clutch within drive system 36 may be used to selectively rotate agitator 42. In other embodiments, separate drive systems may be used to rotate wash basket 34 and agitator 42.

A water inlet 44 may be provided to dispense water into wash tub 16. In some embodiments, for example, hot and cold valves 46, 48 may be coupled to external hot and cold water supplies through hot and cold inlets 50, 52, and may output to one or more nozzles 54 to dispense water of varying temperatures into wash tub 16. In addition, a pump system 56, e.g., including a pump and an electric motor, may be coupled between a low point, bottom or sump in wash tub 16 and an outlet 58 to discharge greywater from wash tub 16. In some embodiments, it may be desirable to utilize multiple nozzles 54, and in some instances, oscillating nozzles 54, such that water dispensed into the wash tub is evenly distributed over the top surface of the load.

In some embodiments, laundry washing machine 10 may also include a dispensing system 60 configured to dispense detergent, fabric softener and/or other wash-related products into wash tub 16. Dispensing system 60 may be configured in some embodiments to dispense controlled amounts of wash-related products, e.g., as may be stored in a reservoir (not shown) in laundry washing machine 10. In other embodiments, dispensing system 60 may be used to time the dispensing of wash-related products that have been manually placed in one or more reservoirs in the machine immediately prior to initiating a wash cycle. Dispensing system 60 may also, in some embodiments, receive and mix water with wash-related products to form one or more wash liquors that are dispensed into wash tub 16. In still other embodiments, no dispensing system may be provided, and a user may simply add wash-related products directly to the wash tub prior to initiating a wash cycle.

It will be appreciated that the particular components and configuration illustrated in FIG. 3 are typical of a number of common laundry washing machine designs. Nonetheless, a wide variety of other components and configurations are used in other laundry washing machine designs, and it will be appreciated that the herein-described functionality generally may be implemented in connection with these other designs, so the invention is not limited to the particular components and configuration illustrated in FIG. 3.

Further, to support various automated functionality described herein, laundry washing machine 10 may also include one or more sensors. One or more force sensors 62, for example, may be used to sense the mass or weight of the contents of the wash tub. In the illustrated embodiment, for example, each force sensor 62 may be implemented as a load cell coupled to one of the suspension rods 30, or alternatively on other structures supporting the wash tub, e.g., a leg, spring or damper. Each load cell may be an electro-mechanical sensor that outputs a signal that varies with a displacement based on load or weight, and thus outputs a signal that varies with the weight of the contents of wash tub 16. In other embodiments, other types of transducers or sensors that generate a signal that varies with applied force, e.g., strain gauges, may be used. Furthermore, in other embodiments, load cells, or other appropriate transducers or sensors, may be positioned elsewhere in a laundry washing machine to generate a plurality of signals that vary in response to the weight of the contents of wash tub 16. In some embodiments, for example, transducers may be used to support an entire load washing machine, e.g., a plurality of legs of a machine. Other types and/or locations of transducers suitable for generating a signal that varies with the weight of the contents of a wash tub will be apparent to one of ordinary skill in the art having the benefit of the instant disclosure. In addition, in some embodiments, a force sensor may also be used for vibration sensing purposes, e.g., to detect excessive vibrations resulting from an out-of-balance load. In other embodiments, however, no vibration sensing may be used, while in other embodiments, separate sensors may be used to sense vibrations.

A fluid level sensor may be used to generate a signal that varies with the level or height of fluid in wash tub 16. In the illustrated embodiment, for example, a fluid level sensor may be implemented using a pressure sensor 64 in fluid communication with a low point, bottom or sump of wash tub 16 through a tube 66 such that a pressure sensed by pressure sensor 64 varies with the level of fluid within the wash tub. It will be understood that the addition of fluid to the wash tub will generate a hydrostatic pressure within the tube that varies with the level of fluid in the wash tub, and that may be sensed, for example, with a piezoelectric or other transducer disposed on a diaphragm or other movable element. It will be appreciated that a wide variety of pressure sensors may be used to provide fluid level sensing, including, among others, combinations of pressure switches that trigger at different pressures. It will also be appreciated that fluid level in the wash tub may also be sensed using various non-pressure based sensors, e.g., optical sensors, laser sensors, etc.

Additional sensors may also be incorporated into laundry washing machine 10, e.g., turbidity, conductivity, and/or flow sensors. In addition, in some embodiments, a camera or other image sensor 68 may be used, for example, to sense the colors of items in a load to be washed by laundry washing machine 10, or to sense other aspects of a load placed in the wash tub. In some in instances, image sensor 68 may be located proximate an opening of the wash tub 16, facing down into the wash tub. In other embodiments, however, image sensor 68 may be oriented generally upwardly facing and/or may be positioned elsewhere in the laundry washing machine, e.g., on a door, proximate a top edge of a door on a front-load laundry washing machine, and in other suitable locations. In addition, in some embodiments, multiple image sensor may be used to capture the wash tub from different viewpoints.

As will become more apparent below, one or more image sensors 68 may also be used to capture images of the wash tub for use in connection with determining a level state of the wash tub, and thus of the laundry washing machine itself. The image sensor(s) used in level state determination may be the same image sensor(s) used for sensing a load in some embodiments, while in other embodiments, different image sensors may be used. An image sensor, in this regard, may be considered to include any type of device or sensor capable of capturing electromagnetic radiation from a wash tub. In many instances, the electromagnetic radiation may be visible light, although it will be appreciated that captured image data in other embodiments may include data captured in other spectra, e.g., infrared and/or ultraviolet spectra. To facilitate image capture, one or more emitters 74, e.g., one or more LED lights, may be used to illuminate the wash tub during image capture.

In addition, in some embodiments, one or more legs 70 of laundry washing machine 10 may include height adjusters 72 to assist in leveling the laundry washing machine. Height adjusters 72 may be manually activated in some embodiments, e.g., by spinning the legs to turn respective threaded shafts that vary how far the legs project below the laundry washing machine, while in other embodiments height adjusters 72 may be electromechanical in nature, and controllable by a controller of the laundry washing machine to automatically raise and lower the legs.

Now turning to FIG. 4, laundry washing machine 10 may be under the control of a controller 80 that receives inputs from a number of components and drives a number of components in response thereto. Controller 80 may, for example, include one or more processors 82 and a memory 84 within which may be stored program code for execution by the one or more processors. The memory may be embedded in controller 80, but may also be considered to include volatile and/or non-volatile memories, cache memories, flash memories, programmable read-only memories, read-only memories, etc., as well as memory storage physically located elsewhere from controller 80, e.g., in a mass storage device or on a remote computer interfaced with controller 80.

As shown in FIG. 4, controller 80 may be interfaced with various components, including the aforementioned drive system 36, hot/cold inlet valves 46, 48, pump system 56, dispenser 60, force sensors 62, fluid level sensor 64, image sensor(s) 68 (as well as any associated emitters), and height adjusters 72 (if electromechanically implemented). In addition, controller 80 may be interfaced with additional components such as a door switch that detects whether door 12 is in an open or closed position and a door lock that selectively locks door 12 in a closed position. Moreover, controller 80 may be coupled to a user interface 86 including various input/output devices such as knobs, dials, sliders, switches, buttons, lights, textual and/or graphics displays, touch screen displays, speakers, image capture devices, microphones, etc. for receiving input from and communicating with a user. In some embodiments, controller 80 may also be coupled to one or more network interfaces 88, e.g., for interfacing with one or more external devices via wired and/or wireless networks such as Ethernet, Bluetooth, NFC, cellular and other suitable networks. Additional components may also be interfaced with controller 80, as will be appreciated by those of ordinary skill having the benefit of the instant disclosure. Moreover, in some embodiments, at least a portion of controller 80 may be implemented externally from a laundry washing machine, e.g., within a mobile device, a cloud computing environment, etc., such that at least a portion of the functionality described herein is implemented within the portion of the controller that is externally implemented.

In some embodiments, controller 80 may operate under the control of an operating system and may execute or otherwise rely upon various computer software applications, components, programs, objects, modules, data structures, etc. In addition, controller 80 may also incorporate hardware logic to implement some or all of the functionality disclosed herein. Further, in some embodiments, the sequences of operations performed by controller 80 to implement the embodiments disclosed herein may be implemented using program code including one or more instructions that are resident at various times in various memory and storage devices, and that, when read and executed by one or more hardware-based processors, perform the operations embodying desired functionality. Moreover, in some embodiments, such program code may be distributed as a program product in a variety of forms, and that the invention applies equally regardless of the particular type of computer readable media used to actually carry out the distribution, including, for example, non-transitory computer readable storage media. In addition, it will be appreciated that the various operations described herein may be combined, split, reordered, reversed, varied, omitted, parallelized and/or supplemented with other techniques known in the art, and therefore, the invention is not limited to the particular sequences of operations described herein.

Image-Based Wash Tub Level Sensing

As noted above, the level state of a laundry washing machine can significantly impact the performance of the machine, as an unlevel machine can increase vibrations, noise, and wear on bearings and other kinetic components when a wash basket is spun during wash, rinse and/or spin operations. Excessive vibrations can also lead to out-of-balance events, which can lengthen wash/spin cycles, or cause those cycles to be interrupted.

Traditionally, laundry washing machines have been provided with height adjustable legs, and users are encouraged to adjust the legs such that all legs firmly touch the ground, and such that the housing of the laundry washing machine is level. The level state of the machine is typically determined using a separate bubble level that is placed on the housing, and the legs are adjusted until the housing of the laundry washing machine is close to level in both forward-to-back and side-to-side directions.

However, leveling a laundry washing machine in such a manner has a number of drawbacks. First, just because the housing of a machine is level does not necessarily mean that the wash tub of the machine is also level. As noted above, a wash tub may be supported in a housing through the use of a suspension assembly, e.g., a set of suspension rods, that allow the wash tub to move relative to the housing to dampen vibrations that are generated when a wash basket within the wash tub is spun. Generally, when the wash tub of the laundry washing machine is level, the springs in the suspension rods will be evenly preloaded, and will be more effective at damping vibrations when the wash basket is spinning. In addition, the axis of rotation of the wash basket will generally be more parallel to the force of gravity, thereby minimizing the effects of gyroscopic precession on the wash basket, and further lessening the amplitude of vibrations when the basket is spinning.

However, due to various manufacturing tolerances, wear in the suspension assembly, etc., a wash tub may not be suspended perfectly level relative to the force of gravity when the housing of the laundry washing machine itself is level. As such, leveling a washing machine using a separate bubble level set on the housing does not always result in the wash tub itself being level.

Second, the level state of a laundry washing machine may change over time, not only due to wear in the suspension assembly, but also due to changes in the location of the laundry washing machine on the floor (e.g., as a result of vibrations over time and/or physically adjusting the location of the laundry washing machine), and even changes in the level state of the floor itself over time. Thus, even if a machine is leveled at initial installation, the level state of the machine may change over time.

Third, some users may simply install a laundry washing machine without leveling the machine as suggested by the manufacturer.

In embodiments consistent with the invention, however, one or more image sensors may be integrated into the laundry washing machine itself and may be used to determine the level state of the laundry washing machine based on captured image data of at least a portion of the wash tub. By doing so, the level state of the machine may be determined at various points after installation (e.g., prior to each wash cycle, on demand in response to user input, etc.), and may be used to alert the user to a potential out-of-level condition, and in some instances, allow a controller to adjust the wash cycle (e.g., to reduce the spin speed) and/or inhibit performance of a wash cycle in response to an out-of-level condition. In addition, in some embodiments, the level state is specifically of the wash tub itself, rather than that of the overall laundry washing machine (as might be determined based on the level state of a surface of the laundry washing machine housing), such that any discrepancies between the wash tub and the laundry washing machine housing (e.g., due to the suspension assembly) may be resolved in favor of leveling the wash tub, i.e., the component that supports the wash basket and many of the other potentially vibration-generating components in the laundry washing machine.

By utilizing an image sensor integrated into a laundry washing machine, a machine may be calibrated in an end-of-line process during manufacturing, and one or more calibration values, referred to herein as tare values, may be determined and stored in a non-volatile memory in the laundry washing machine. The tare values may then be used to calculate a level state of the laundry washing machine during installation, as well as at different times over the life of the machine. In some embodiments, for example, a level state may include a degree and/or angle of level. The degree of level, for example, may represent an amount that the wash tub departs from level, while the angle may represent a direction around the vertical axis that the machine slopes relative to the horizontal plane that is perpendicular to the vertical axis. In some embodiments, for example, the degree of level may be represented using an angle between the vertical axis relative to gravity and the vertical axis of the wash tub and about which the wash basket spins, or alternatively, a distance between the two axes at a predetermined distance from the intersection of those axes (e.g., proximate the location of one or more legs of the machine). The angle of level may also be represented by an angle in some embodiments, e.g., relative to a home position about the vertical axis of the machine, such as the front-to-back direction of the machine. In some embodiments, a level state may also include information regarding the adjustments needed for leveling the wash tub or machine, e.g., one or more corners of the laundry washing machine and the distance the leveling leg(s) at the corner(s) should be adjusted. In other embodiments, the degree of level may be represented using an (X, Y) location in a cartesian plane relative to an origin representing a level state. Other manners of representing the level state may be used in other embodiments, however, so the invention is not limited to these specific representations.

In addition, in some embodiments, an integrated image sensor may be used to perform various other operations in addition to level state determinations. For example, captured image data may be used in various embodiments to determine one or more of load color composition, load type, out-of-balance conditions, excessive foaming, wash fluid level, the presence or absence of a removable agitator, the collapsed/extended state of a collapsible agitator, or child presence detection, among other auxiliary purposes.

Now turning to FIG. 5, and with continuing reference to FIGS. 1-4, in some embodiments, one or more image sensors integrated into a laundry washing machine may be used to capture image data from which a level state of a wash tub, and thus of the laundry washing machine, may be determined. The image sensors are generally desirably disposed at fixed, or at least repeatable, positions and orientations such that images captured during operation are desirably from the same viewpoint, and with substantially the same field of view, as the images captured during calibration. The field of view of each sensor also generally encompasses at least a portion of the interior of the wash tub. Image data, which may be captured by the one or more image sensors, and in some instances, while the wash tub is illuminated, may then be subjected to image processing to determine a relative position of the wash tub. In some embodiments, for example, by analyzing image data from one or more frames captured by an image sensor, the geometry of various features in the wash tub can be detected and used to calculate coordinates of a registration point of the wash tub, and thus measure how far the registration point has deviated from the calibrated location of the registration point for the wash tub. In other embodiments, image processing may be used to correct for image distortion, which in some instances may allow for some variation in image sensor placement. In addition, in some instances, image data may be captured using an external device, e.g., a mobile device, with image distortion correction (and, when appropriate, mobile device accelerometer data) used to accommodate varying positions and orientations of the external device when capturing an image of the wash tub.

In some embodiments, image processing may be performed locally in the laundry washing machine controller, while in other embodiments, at least a portion of the image processing may be performed remote from the laundry washing machine, e.g., in a user's mobile device and/or a remote or cloud-based service in communication with the laundry washing machine. In some embodiments, for example, a laundry washing machine controller may control the image sensor to capture image data, then communicate at least a portion of that data to a remote device to handle image processing, and then receive back level state data, e.g., the coordinates of the registration point of the wash tub, which can then be compared against tare data (e.g., the coordinates of the registration point of the wash tub during calibration) to determine the level state of the laundry washing machine.

Image processing, in some embodiments, may be based on identifying one or more features of the wash tub, including, in some embodiments, features of one or more components housed within the wash tub, e.g., in a wash basket and/or on an agitator and/or impeller. Practically any feature capable of being detected using image processing may be used for determining the level state of the wash tub. For example, FIG. 5 illustrates a wash tub 100, within which is disposed a wash basket 102 and an agitation assembly 104, which includes a relatively flat impeller portion 106 and a vertically-extending tower or agitator portion 108. It will be appreciated that in different machine designs, only an impeller portion or agitator portion may be used, and in some embodiments, a removable or collapsible agitator may be used to enable a user to reconfigure the wash tub for washing different types of loads.

Wash tub 100 may be considered to have a registration point R from which the level state of the wash tub may be determined. In some embodiments, the registration point may be a geometrical centroid for the wash tub, the wash basket and/or the agitation assembly (here, it is all three), although in other embodiments, the registration point may be disposed at different locations, so long as the location is reproduceable between calibration and operational level state determination so that a distance between the calibration location and the captured location can be compared to determine the level state of the wash tub. Orienting the registration point at the centroid, which may also be the axis of rotation for the wash basket and/or the agitation assembly, may be desirable in some embodiments because the relative rotational orientation of these components generally does not need to be determined to ensure that the registration point captured during operation is at the same relative rotational orientation at the registration point captured during calibration.

Various visually distinct features may be utilized in determining the registration point, e.g., ribs 110 on wash basket 104 (one or more of which may be visual distinct from the others to allow for rotational orientation detection, if desired), discs 112 or fins 114 of impeller portion 106, or portions (e.g., fins) of agitator portion 108. In some embodiments, dedicated registration indicia, e.g., the distinctive logo 116 disposed at the top of agitator portion 108, may even be used for registration. The visually distinct features that may be used for registration purposes are practically innumerable, and may include features that are primarily functional in nature as well as features that are specifically designed for registration purposes. Multiple different features, as well as multiple different features of different types, may be used for registration in different embodiments.

It will also be appreciated that, for example, utilizing one or more visually distinct features disposed at a higher elevation of an agitator may also be desirable in some embodiments because tilting of a wash tub may, in some instances, result in a relatively greater change in position of such features in the field of view of the image sensor.

FIG. 6, for example, illustrates an image capture 120 of an impeller 122, as well as the results of image processing based primarily on identifying three discs 124 disposed on the impeller, calculating a centroid of each disc and then calculating a registration point 126 in the geometric center of the triangle 128 formed by the centroids of the discs.

It will be appreciated that a wide variety of image processing techniques, which will be understood by those of ordinary skill in the art having the benefit of the disclosure, may be used to determine the registration point. Various intensity-based, feature-based, model-based and/or learning-based approaches may be used, and combinations of approaches may also be used in some embodiments. In addition, rather than determining registration points during calibration and operation, in some embodiments image captures from calibration and operation may be registered with one another to determine the level state based on the distance between the two image captures. As such, tare data in some embodiments may simply be one or more images captured during calibration, with which image registration of a later captured image may be performed to determine the current level state of the laundry washing machine.

In addition, in some embodiments, a trained machine learning model (illustrated at 90 in FIG. 3), may be used to determine a level state based on captured data. In some embodiments, for example, model 90 may be implemented as a deep neural network (DNN) including an input layer, one or more intermediate layers, and an output layer. In some embodiments, for example, the one or more intermediate layers may include one or more convolutional layers. The dimensions/shape of the input layer may be dependent on the shape of the image data to be applied, while the dimensions/shape of the output layer may be dependent on various factors such as how many class probabilities are to be predicted, among others. In some embodiments, multiple convolution layers may be provided, and max pooling and/or other layers such as affine layers, softmax layers and/or fully connected layers may optionally be interposed between one or more of the convolution layers and/or between a convolution layer and the output layer. Other embodiments may not include any convolution layer and/or not include any max pooling layers, and in still other embodiments, other machine learning models may be used, e.g., Bayesian models, random forest models, Markov models, etc. Such a model may output, for example, a location of the registration point based on one or more input images, which may then be compared against the location determined during calibration. Training data for training such a model may include, for example, image captures made when the laundry washing machine is positioned at different degrees and/or directions of tilt, and with the agitation assembly and wash tub at different relative rotational orientations, and labeled with associated registration point coordinates.

As noted above, it may be desirable to calibrate a laundry washing machine during manufacture to generate calibration or tare values that may be used in future leveling operations. FIG. 7, for example, illustrates an operational sequence 140 for calibrating a laundry washing machine during an end-of-line process during manufacturing. In block 142, a predetermined amount of water may optionally be dispensed in the wash tub to reduce suspension system effects during the calibration process. Alternatively, block 142 may be skipped, and calibration may be performed using an empty wash tub. In either event, the laundry washing machine is supported on a known level surface, and optionally with the leveling legs set at equal heights.

Next, in block 144, one or more images of the wash tub are captured from one or more image sensors of the laundry washing machine, and optionally while the wash tub is illuminated. The captured image data is then processed in block 146, using any of the various techniques described above, and in block 148, X and Y tare values, representing the location of the registration point of the wash tub in the field of view of the image sensor(s) are calculated. The tare values are then stored in a non-volatile memory of the laundry washing machine (e.g., an EEPROM) in block 150, and calibration is complete.

FIG. 8 illustrates an example operational sequence 160 for performing a leveling operation during or after installation of a laundry washing machine. In block 162, a predetermined amount of water is optionally dispensed into the wash tub to reduce suspension system effects, in a similar manner to that described above in connection with FIG. 7. The same amount of water as used during calibration, or a different amount of water, may be dispensed in different embodiments, and in some embodiments, block 162 may be skipped, and leveling may be performed using an empty wash tub.

Next, in block 164, one or more images of the wash tub are captured from one or more image sensors of the laundry washing machine, and optionally while the wash tub is illuminated. The image data is then processed in block 166, using any of the various techniques described above, and in block 168, the relative level state of the laundry washing machine is determined. The relative level state, for example, may be based on a comparison of the current X, Y location of the registration point of the wash tub in the field of view of the image sensor(s), which may be determined based on the aforementioned image processing, and the location of the registration point during calibration and provided in the form of the X and Y tare values. The relative level state, in some embodiments, may include a distance and direction between the current and calibration X, Y locations, although other representations of the level state may be used, as discussed above.

Next, level state is tested to determine if it meets a level criterion in block 170, e.g., if the level state is within a predetermined tolerance from a perfectly level state, and if it is, the leveling operation is complete. If not, however, block 170 passes control to block 172, where the results are displayed to the user.

Next, as represented by blocks 174 and 176, manual adjustments of one or more manual height adjusters by a user and/or automatic adjustments of one or more electromechanical height adjusters by the controller may be performed, and control returns to block 164 to capture new image data. As such, the level state of the laundry washing machine may be determined dynamically as a user and/or the controller adjusts the height adjusters until a sufficiently leveled condition is obtained.

While a number of different displays may be used in different embodiments, FIG. 9 illustrates one example display 180 that simulates a bullseye-type bubble level, where the current level state of the laundry washing machine is represented using a first icon 182 disposed at coordinates corresponding to the (X, Y) values calculated above, and with the origin (representing a perfectly leveled condition) represented using a second icon 184. A user may be presented with various guides, e.g., an arrow 186 and/or textual instructions 188 that indicate which height adjuster(s) should be adjusted and how much. As the height adjusters are adjusted, icon 182 will move towards icon 184, and a leveled condition will be represented when icon 182 overlaps with icon 184, as represented at position 182′.

Other manners of displaying the current level state of a laundry washing machine, its relationship to a leveled condition, and in some instances, the instructions for obtaining the leveled condition, may be used in different embodiments, including other textual and/or graphical displays, audible cues, vocal instructions, etc. In addition, the presentation of such information may be made on the laundry washing machine user interface and/or on the user interface of an external device, e.g., a mobile device running a mobile app in communication with the laundry washing machine.

Next, turning to FIG. 10, once the laundry washing machine is leveled, the image sensor(s) may be used to periodically check the level state of the laundry washing machine, as well as perform other operations, as will be discussed in greater detail below. In particular, FIG. 10 illustrates an operational sequence 200 for performing a wash cycle using the laundry washing machine. At the start of the wash cycle, image data may be captured in block 202, and the image data may be used to determine a load type of the load contained in the wash tub (e.g., via analysis of color composition data captured from the load as it is placed in the wash tub) in block 204. In addition, in block 206 a level state of the laundry washing machine may be determined using the image data, and block 208 may determine if a level error criterion has been met. The level error criterion may be configured to inhibit initiation of a wash cycle if it is determined that the level state is excessively unlevel, and as such, if the criterion is met, control may pass to block 210 to notify the user (e.g., via a display, a mobile app, a notification, etc.) of the need to level the laundry washing machine.

If the level error criterion is not met, however, control passes to block 212 to determine a spin profile for the wash cycle. For example, where the level state of the laundry washing machine is not so unlevel as to trigger the level error criterion but is still relatively unlevel to where higher speed spin operations could lead to knocking, excessive vibrations, or out-of-balance conditions, it may be desirable to reduce the maximum spin speed during the wash cycle. In contrast, if the level state is close to a perfectly leveled condition, higher speed spin operations may be justified due to a reduced risk of excessive vibrations.

Next, in block 214, additional wash cycle parameters are determined, e.g., wash and/or rinse operation types, durations, repetitions, etc., using various operations that will be apparent to those of ordinary skill in the art having the benefit of the instant disclosure. At least some of these parameters may be determined at least in part using the load type determined in block 204. Then, in block 216, the wash cycle is performed, using the various parameters and profiles determined above. In addition, image data may be captured at various points in the wash cycle, e.g., to detect an out of balance load, e.g., in response to detecting force variations above a predetermined threshold, as well as other load conditions such as excessive foaming, wash fluid level, etc. Thus, it will be appreciated that the captured image data may be used for purposes beyond determining a level state of the laundry washing machine in some embodiments.

It will be appreciated that, while certain features may be discussed herein in connection with certain embodiments and/or in connection with certain figures, unless expressly stated to the contrary, such features generally may be incorporated into any of the embodiments discussed and illustrated herein. Moreover, features that are disclosed as being combined in some embodiments may generally be implemented separately in other embodiments, and features that are disclosed as being implemented separately in some embodiments may be combined in other embodiments, so the fact that a particular feature is discussed in the context of one embodiment but not another should not be construed as an admission that those two embodiments are mutually exclusive of one another. Various additional modifications may be made to the illustrated embodiments consistent with the invention. Therefore, the invention lies in the claims hereinafter appended.