Matrix operations in an integrated circuit device

Description

BACKGROUND OF THE INVENTION

This invention relates to performing matrix operations in integrated circuit devices, and particularly in programmable integrated circuit devices such as programmable logic devices (PLDs).

Matrix operations on large matrices are becoming more common. For some technical problems, solutions may involve matrices as large as 256-by-256. For example, it may be necessary to factor and/or invert a large matrix, or to solve for the eigenvalues of a large matrix. However, while a vector-based approach may be useful for performing such operations, providing the resources for such large vector-based solutions may be problematic, particularly where the size of the matrices may differ. This may particularly be the case in programmable devices, where different users may require resources for vector-based matrix operations of different sizes.

SUMMARY OF THE INVENTION

The present invention relates to circuitry for performing vector-based matrix operations for matrices of arbitrary size, up to a predetermined maximum size. The circuitry can be provided in a fixed logic device, or can be configured into a programmable integrated circuit device such as a programmable logic device (PLD).

The present invention may be used where all input row vectors of a matrix are to be combined—e.g., by multiplication—with the same initial vector, which may be one of the rows. Column memories may be provided to hold the input matrix data, so that each row may be read by simultaneously accessing the same index in each column memory. In accordance with the invention, the number of column memories may be less than the actual number of columns so that multiple physical “row access” operations are used to access each logical row. A “circular latch” may be provided to hold the initial vector.

The size of the circular latch may be matched to that of the column memories in that the circular latch may include a plurality of rows each having the same number of memories as there are column memories, with the number of rows corresponding to the maximum number of “row access” operations needed per row of the input matrix, which may vary from row to row, even within the same matrix, as discussed below. In a fixed-logic device, the total number of memories may be determined in advance based on the intended application. In a programmable device, there may be no limit, as the memories from which the circular latch is constructed may be freely available as parts of the logic elements of the device. Alternatively, a programmable device may be provided with a number of dedicated memories available for configuration as part of the circular latch, so that user configurations would be limited by that number.

Therefore, in accordance with the present invention, there is provided matrix operations circuitry for performing a vector operation on an input matrix having a first number of columns, where the vector operation includes operations combining one row of the matrix and each row of the matrix. The matrix operations circuitry includes a first set of a second number of column memories, where the second number is smaller than the first number, so that at least one row of the input matrix is stored in multiple rows of the column memories. The matrix operations circuitry also includes a vector operations circuit that performs the operations combining a selected row of the matrix and each row of the matrix, and a first set of first input registers equal in number to the second number, for inputting each row to the vector operations circuit from the column memories. The matrix operations circuitry further includes a first set of second input registers equal in number to the second number, for inputting that one row to the vector operations circuit, a first circular latch for storing that one row, and selection circuitry for circulating values from the selected row between the first set of second input registers and the first circular latch.

A method of configuring such circuitry on a programmable device, and a machine-readable data storage medium encoded with software for performing the method, are also provided.

BRIEF DESCRIPTION OF THE DRAWINGS

Further features of the invention, its nature and various advantages will be apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:

FIG. 1 shows an example resultant matrix of a Cholesky decomposition operation;

FIG. 2 shows one embodiment of a datapath arrangement for Cholesky decomposition;

FIG. 3 shows one embodiment of a circuit arrangement used in the performance of Cholesky decomposition;

FIG. 4 shows one embodiment of an input datapath circuit arrangement in accordance with the present invention;

FIGS. 5A and 5B (hereinafter collectively referred to as “FIG. 5”) show a logical representation of memory organized in accordance with an embodiment of the present invention;

FIG. 6 shows a physical representation of memory organized in accordance with an embodiment of the present invention;

FIGS. 7A and 7B (hereinafter collectively referred to as “FIG. 7”) show a circuit arrangement used in the performance of Cholesky decomposition in accordance with an embodiment of the present invention;

FIG. 8 shows a physical representation of memory organized in accordance with another embodiment of the present invention;

FIG. 9 is a cross-sectional view of a magnetic data storage medium encoded with a set of machine-executable instructions for performing the method according to the present invention;

FIG. 10 is a cross-sectional view of an optically readable data storage medium encoded with a set of machine executable instructions for performing the method according to the present invention; and

FIG. 11 is a simplified block diagram of an illustrative system employing a programmable logic device incorporating the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention may be used to configure circuitry for any class of matrix operations involving multiplication of a series of vectors by a single initial vector. For purposes of illustration, an embodiment of the invention will be described in connection with the Cholesky decomposition of a triangulated matrix—i.e., a square matrix having no values (or all zero values) above the diagonal—such as that described in copending, commonly-assigned U.S. patent application Ser. No. 12/557,846, filed Sep. 11, 2009, which is hereby incorporated by reference herein in its entirety.

An example 100 of a triangulated matrix l resulting from a lower-form Cholesky decomposition is shown in FIG. 1. Although the size of matrix l may differ, matrix l will always be a square matrix. In this case, matrix l is a 6-by-6 matrix. The elements on the diagonal are l₁₁, . . . , l₆₆. In each jth column, the elements under l_jj are l_ij, i=j₊₁, . . . , i_max (in this case, i_max=6). The matrix may be considered to be empty above the diagonal, or the elements above the diagonal may be considered to be zeroes.

As described in the above-incorporated copending application, each element l_ij can be calculated using two datapaths. The first datapath calculates the following result:
l_x=a_x−L_x,L_x
where for l and a, x=ij; for the L vectors, x=i or j, respectively; and L_x,L_x denotes the inner product of the L vectors.

The first output (x=jj) of the first datapath is latched at the input of a second datapath, which calculates the actual l_ij. The first element of the column (l_jj) is calculated as the inverse square root of the input (a_jj−L_j,L_j), multiplied by the input, generating the square root of the input. The inverse square root is used instead of a direct square root calculation, because it can be reused for the following elements in the column using multiplication, which is easier to implement than division.

To calculate all of the subsequent values in the column, the latched first datapath output is used for the inverse square root input which is a first multiplier input, and the other multiplier input is, for each subsequent term, the corresponding output of the first datapath. The entire column can therefore be calculated without waiting for any individual element to be finished.

FIG. 2 shows how the matrix values are stored as described in the above-incorporated copending application. Each a_ij value is a single number that can be addressed in a single clock cycle, but each L_i or L_j row vector is j−1 numbers which would require j−1 clock cycles to address if all values were stored in a single memory. However, in accordance with the above-incorporated copending application, matrix a may be stored in a single memory 201, while each column of matrix l may be stored in one of a plurality of i_max separate memories 202. The ith element of each of the separate column memories can be addressed simultaneously, allowing the entire row vector to be read out within a single clock cycle. This may be referred to as a “column-wise” memory architecture.

For example, programmable logic devices available from Altera Corporation, of San Jose, Calif., may have a smaller number of larger memory blocks (e.g., 144 kb memory blocks), one of which could be used as memory 201 to store matrix a, and a larger number of smaller memory blocks (e.g., 9 kb memory blocks), i_max of which could be used as memories 202 to separately store the columns of matrix l. Of course, it is not necessary to use different sizes of memories for memories 201, 202; if a sufficient number of larger memories is available, any one or more of the memories used as column memories 202 to separately store the columns of matrix l may be the same size as (or even larger than) the memory used as memory 201 to store matrix a.

Thus, in a single clock cycle, address input 211 may be applied to memory 201 to read out matrix element a_ij at 221 for input to calculation datapath 300, while address input 212 may be applied to the appropriate j−1 memories 202 on path 203 to read out vector L_i, and address input 222 may be applied to the appropriate j−1 memories 202 on path 213 to read out vector L_j. The outputs 221, 203, 213 maybe input to calculation datapath 300, described in more detail in connection with FIG. 3, which outputs the individual l_ij values at 204, and also feeds each back at 205 into the respective jth column memory 202.

Datapath 300, which may be implemented in fixed or programmable logic, includes inner product datapath 301 and inverse square root datapath 302.

Inner product datapath 301 includes inner product generator 311 and subtractor 321 to subtract the inner product from a_ij. Inner product generator 311 may include a sufficient plurality of multipliers and adders to simultaneously multiply i_max pairs of values, and then add those products together. For complex vectors, inner product generator 311 may include sufficient multipliers and adders to simultaneously multiply 2 (i_max) pairs of values, and also may include the necessary components to compute the complex conjugate values for L_j in the case where the values are complex. The L_j term is latched in register 331 at the beginning of a column process and is not changed until the next column is started.

Starting with the second column, the first output of inner product datapath 301 for each column—i.e., each l_jj—is latched into register 312 as the input to inverse square root datapath 302 for the duration of calculation of that column. Inverse square root datapath 302 includes inverse square root module 322 for calculating the inverse square root of l_jj, and multiplier 332 for multiplying the inverse square root by the current l_ij. The latching of l_jj into register 312 delays its input to multiplier 332 by one clock cycle. Therefore, the input of l_ij to multiplier 332 also is delayed, by register 342, so that latency is the same for both inputs.

For the first column, terms are generated using division. The top term, l₁₁ is a₁₁^0.5 and all the subsequent inputs for the first column are divided by the top term so that l_i1=a_i1/a₁₁^0.5=a_i1×a₁₁^−0.5. This is accomplished using multiplexer 350 to allow the a_ij inputs 351 to bypass inner product datapath 301.

In the example of FIGS. 1-3, as described in the above-incorporated copending application, the matrix is a relatively small 6-by-6 matrix. However, those techniques may become impractical as matrices become larger. Accordingly, the present invention provides circuitry, and a technique, that is useful even with larger matrices, as well as with smaller matrices.

In accordance with the present invention, one can define matrix size to refer to the number of rows and columns in the matrix. For a given matrix size, the user may choose a given vector processing size. The vector processing size is the number of elements of a row of the input matrix that are processed at one time, although it should be noted that for some applications, the invention may be implemented on a column basis instead of a row basis. Thus, for each row (or column), there will be a number of partial inner product results. A parallel adder may be used to sum those partial inner product results. The adder size is determined based on the number of partial results to be added.

In an embodiment implemented in a programmable logic device, programming software for the programmable logic device may include a library containing a set of precompiled circuit configurations for vector multiplication blocks and adders of varying sizes that may be needed for matrices of different sizes and different vector processing sizes.

A number of data memories may be provided, and may be accessed in parallel to extract a portion of a row of the matrix in a single clock cycle. The number of data memories preferably is the same as the vector processing size. For real matrices, only a single set of memories is needed, while for complex matrices, a separate set of real and imaginary memories may be used. As an alternate embodiment in the complex case, the real and imaginary portions of any element could be stored in a single larger memory—e.g., having twice the data width.

As described above in connection with FIGS. 1-3, each element of a column is determined based on the inner products of the corresponding row vectors. The same is true in embodiments of the present invention. The first column may processed as it is input or, alternatively, after it is loaded.

FIG. 4 shows a block diagram of the datapath 400 of the input block for the Cholesky decomposition example, in which data for the first row is used in the square root calculation described above. As shown, datapath 400 includes a square root processing block 401 which may be used on the first-row data before they are input to memories 701 (FIG. 7). During input of data for subsequent rows, the data bypass square root processing block 401 and are input directly into memories 701. Delays 404 may be provided to compensate for the latency of square root processing block 401 in processing the first row. The first value (matrix index {0,0}) may be latched into the denominator input 402 of square root processing block 401 for reuse. Even for a complex matrix, this value will be real (as are all the values on the diagonal), so only a real component needs to be latched. That first value, and all subsequent values of the first column are also input into the numerator 403 of square root processing block 401. Square root processing block 401 multiplies the inverse square root of the denominator input 402 with the complex numerator input 403. As the denominator input is always real, one real inverse square root component and two real multipliers are required. An identical square root processing block is used elsewhere in the main part of the calculation as described below, and it is also possible to reuse the same square root processing block for both parts of the calculation, by providing appropriate multiplexing.

FIGS. 5 and 6 illustrate the example of a 12-by-12 matrix with a vector processing size of four, which means that each row is represented by a 4-by-3 submatrix. This logical representation 500 may be seen in FIG. 5 where each row is represented by a respective submatrix 501. FIG. 6 shows the physical storage of that same data in a memory 600, illustrated in the form of a table. In the example of FIGS. 5 and 6, the input matrix is triangulated, so not all rows completely fill their respective submatrices. However, the adder size has to be able to accommodate the largest submatrices which, in this example, would require three separate accesses, so the adder size in this example is three. In alternative embodiments the matrix size might not be the product of the vector processing size and the adder size, but the submatrices preferably would still have a number of rows equal to the vector processing size and a number of columns equal to the adder size.

As seen in FIG. 6, a vector processing size of four means that four memories 601, 602, 603, 604 would be used, each containing a number of columns from the input matrix. In this example, the first memory 601 contains columns 0, 4, and 8, as can be seen from the column indices of the individual memory elements.

FIG. 7 shows an architecture 700 according to an embodiment of the invention which may, for example, be used for the Cholesky decomposition described above. In this example, the number of input column memories 701 is three, rather than four, but this is exemplary only. Memories 701 are provided in two groups 711, 721 for processing the real and imaginary parts, respectively.

Each partial row is read from the column memories 701 into the memories 702. The partial row data for the first row also are read into memories 703 at the outputs of circular latches 716, 717 (only one of which is needed if imaginary values are not being used). ctrl_valid_term signal 706 may be used to zero out any contribution from a column memory that does not contain valid data (e.g., from positions corresponding to the upper, empty half of a triangulated matrix). Memories 702 function as staging registers to (1) break up the routing path for higher system speeds, and (2) equalize the delays into vector processing block 707 from memories 701 and from circular latches 716, 717

The vector multiplication processing for the inner product (for the real part, and also, if present, for the imaginary part) may be carried out in vector processing block 707. The partial results may be delayed by pipeline registers 708 and combined in registers 709. Gates 710, under control of ctl_valid_vector signal 731, prevent data in pipeline registers 708 that is not to be added (e.g., data from empty subrows in FIG. 5) from passing to registers 709.

For this Cholesky decomposition example, sum processing block 713 adds the reconstituted inner product(s) 712 and subtracts that sum from the current a_ij/a_jj, which is selected from the row vector for the current output column by multiplexer 714, 715 for the real and imaginary paths, respectively. In the first column of each group, the a_ij/a_jj value will be the first element in the last row of the current sub-matrix.

The first row may be processed during or just after loading, with the same values in registers 702 and 703. The values from registers 703 are then multiplexed into the correct row of respective circular latch 716, 717 (only one would be used if there is no imaginary part). For example, the first sub-row may be stored into row 726 of circular latch 716, 717, the second sub-row may be stored into row 727, the third sub-row may be stored into row 728, etc.

For the second and subsequent rows, the input column memories 701 may be accessed in parallel and read into memories 702. The respective sub-row from circular latch 716, 717 is read into memories 703. After the values in memories 702, 703 are entered into vector processing block 707, the remaining values in circular latch 716, 717 move down one row, with the next sub-row now in memories 703, and the values previously in memories 703 are circulated back to the topmost used row (based on the adder size of the current configuration) of circular latch 716, 717. The latch control signal preferably is aligned with the start of the column as it only happens once per column. With each subsequent group, the latch signal is advanced one clock less. Each circular latch 716, 717 is clocked at the same rate as the reads from the input column memories 701, and therefore the same indexed sub-row of the latched vector would be aligned with the current indexed sub-row of the current input vector.

The a_ij/a_jj term may be delayed by delay 718, 719 (again, there is only one if complex numbers are not involved) so that it arrives at the input of sum processing block 713 at the same time as when the current set of vector products 712.

The first result of sum processing block 713 may be latched at the input of square root processing block 720, and then each sum result (including the first one) is streamed through square root processing block 720, to generate the individual elements of the current output column. As noted above, a single square root processing block can be used as both square root processing block 701 and square root processing block 720, by providing appropriate multiplexing (not shown).

The physical memory structure shown in FIG. 6 can be further optimized for area, by eliminating the unused rows, with the trade-off of more complex addressing requirements. An optimized memory map 750 for a 12-by-12 matrix is shown in FIG. 8. For larger matrices approaching 256-by-256, such optimization can save a considerable amount of memory. Indeed, typical savings may be on the order of about 40%.

Although described above in the context of Cholesky decomposition, the present invention may be implemented in other embodiments for a variety of vector-based matrix operations. Therefore, in some of those embodiments, some of the structures included in with the embodiments described above, such as sum processing block 713, or square root processing blocks 401, 720, may not be included, but those embodiments would still be within the present invention.

The structures described above may be provided in fixed logic, in which case the sizes of the various computational components may be fixed to a particular application. Alternatively, the fixed logic circuitry could allow for limited parameterization. For example, circular latches of a certain size may be provided, but it may be possible to select how many rows are used, allowing the circuitry to be used for matrices up to that size.

Another potential use for the present invention may be in programmable integrated circuit devices such as programmable logic devices, where programming software can be provided to allow users to configure a programmable device to perform matrix operations. Where the programmable device is provided with a certain number of dedicated blocks for arithmetic functions (to spare the user from having to configure arithmetic functions from general-purpose logic), many of the computational components may be configured from those blocks. Others may be configured from the programmable resources. For example, memories associated with different individual logic elements may be connected to form the aforementioned circular latches.

Instructions for carrying out a method according to this invention for programming a programmable device to perform matrix decomposition may be encoded on a machine-readable medium, to be executed by a suitable computer or similar device to implement the method of the invention for programming or configuring PLDs or other programmable devices to perform addition and subtraction operations as described above. For example, a personal computer may be equipped with an interface to which a PLD can be connected, and the personal computer can be used by a user to program the PLD using a suitable software tool, such as the QUARTUS® II software available from Altera Corporation, of San Jose, Calif.

FIG. 9 presents a cross section of a magnetic data storage medium 800 which can be encoded with a machine executable program that can be carried out by systems such as the aforementioned personal computer, or other computer or similar device. Medium 800 can be a floppy diskette or hard disk, or magnetic tape, having a suitable substrate 801, which may be conventional, and a suitable coating 802, which may be conventional, on one or both sides, containing magnetic domains (not visible) whose polarity or orientation can be altered magnetically. Except in the case where it is magnetic tape, medium 800 may also have an opening (not shown) for receiving the spindle of a disk drive or other data storage device.

The magnetic domains of coating 802 of medium 800 are polarized or oriented so as to encode, in manner which may be conventional, a machine-executable program, for execution by a programming system such as a personal computer or other computer or similar system, having a socket or peripheral attachment into which the PLD to be programmed may be inserted, to configure appropriate portions of the PLD, including its specialized processing blocks, if any, in accordance with the invention.

FIG. 10 shows a cross section of an optically-readable data storage medium 810 which also can be encoded with such a machine-executable program, which can be carried out by systems such as the aforementioned personal computer, or other computer or similar device. Medium 810 can be a conventional compact disk read-only memory (CD-ROM) or digital video disk read-only memory (DVD-ROM) or a rewriteable medium such as a CD-R, CD-RW, DVD-R, DVD-RW, DVD+R, DVD+RW, or DVD-RAM or a magneto-optical disk which is optically readable and magneto-optically rewriteable. Medium 810 preferably has a suitable substrate 811, which may be conventional, and a suitable coating 812, which may be conventional, usually on one or both sides of substrate 811.

In the case of a CD-based or DVD-based medium, as is well known, coating 812 is reflective and is impressed with a plurality of pits 813, arranged on one or more layers, to encode the machine-executable program. The arrangement of pits is read by reflecting laser light off the surface of coating 812. A protective coating 814, which preferably is substantially transparent, is provided on top of coating 812.

In the case of magneto-optical disk, as is well known, coating 812 has no pits 813, but has a plurality of magnetic domains whose polarity or orientation can be changed magnetically when heated above a certain temperature, as by a laser (not shown). The orientation of the domains can be read by measuring the polarization of laser light reflected from coating 812. The arrangement of the domains encodes the program as described above.

A PLD 90 programmed according to the present invention may be used in many kinds of electronic devices. One possible use is in a data processing system 900 shown in FIG. 11. Data processing system 900 may include one or more of the following components: a processor 901; memory 902; I/O circuitry 903; and peripheral devices 904. These components are coupled together by a system bus 905 and are populated on a circuit board 906 which is contained in an end-user system 907.

System 900 can be used in a wide variety of applications, such as computer networking, data networking, instrumentation, video processing, digital signal processing, or any other application where the advantage of using programmable or reprogrammable logic is desirable. PLD 90 can be used to perform a variety of different logic functions. For example, PLD 90 can be configured as a processor or controller that works in cooperation with processor 901. PLD 90 may also be used as an arbiter for arbitrating access to a shared resources in system 900. In yet another example, PLD 90 can be configured as an interface between processor 901 and one of the other components in system 900. It should be noted that system 900 is only exemplary, and that the true scope and spirit of the invention should be indicated by the following claims.

Various technologies can be used to implement PLDs 90 as described above and incorporating this invention.

It will be understood that the foregoing is only illustrative of the principles of the invention, and that various modifications can be made by those skilled in the art without departing from the scope and spirit of the invention. For example, the various elements of this invention can be provided on a PLD in any desired number and/or arrangement. One skilled in the art will appreciate that the present invention can be practiced by other than the described embodiments, which are presented for purposes of illustration and not of limitation, and the present invention is limited only by the claims that follow.