HDF5 Algebra

1 Overview

Algebra and matrix factorizations over HDF5 datasets (PCA, SVD, QR, Cholesky, eigen, solve, pseudoinverse).

2 Functions

2.1 bdCholesky_hdf5

Computes the Cholesky decomposition of a symmetric positive-definite matrix stored in an HDF5 file. The Cholesky decomposition factors a matrix A into the product A = LL’ where L is a lower triangular matrix.

2.2 bdQR_hdf5

Computes the QR decomposition of a matrix stored in an HDF5 file, factoring it into a product A = QR where Q is an orthogonal matrix and R is an upper triangular matrix. Results are stored back in the HDF5 file.

2.3 bdSVD_hdf5

Computes the Singular Value Decomposition (SVD) of a large matrix stored in an HDF5 file. The SVD decomposes a matrix A into a product A = UDV’ where U and V are orthogonal matrices and D is a diagonal matrix containing the singular values.

2.4 bdSolve_hdf5

Solves the linear system AX = B where matrices A and B are stored in HDF5 format. The solution X is written back to the HDF5 file.

2.5 bdPCA_hdf5

Performs Principal Component Analysis (PCA) on a large matrix stored in an HDF5 file. PCA reduces the dimensionality of the data while preserving as much variance as possible. The implementation uses SVD internally for efficient and numerically stable computation.

2.6 bdCrossprod_hdf5

bdCrossprod_hdf5

2.7 bdtCrossprod_hdf5

bdtCrossprod_hdf5

2.8 bdCreate_diagonal_hdf5

Creates a diagonal matrix or vector directly in an HDF5 file using block-wise processing to minimize memory usage. This unified function replaces separate diagonal and identity matrix creation functions, providing flexible diagonal creation with automatic parameter detection.

2.9 bdDiag_add_hdf5

Performs optimized diagonal addition between two datasets stored in HDF5 format. Automatically detects whether inputs are matrices (extracts diagonals) or vectors (direct operation) and uses the most efficient approach.

2.10 bdDiag_subtract_hdf5

Performs optimized diagonal subtraction between two datasets stored in HDF5 format. Automatically detects whether inputs are matrices (extracts diagonals) or vectors (direct operation) and uses the most efficient approach. This function is ~50-250x faster than traditional matrix operations for diagonal computations.

2.11 bdDiag_multiply_hdf5

Performs optimized diagonal multiplication between two datasets stored in HDF5 format. Automatically detects whether inputs are matrices (extracts diagonals) or vectors (direct operation) and uses the most efficient approach. This function performs element-wise multiplication and is ~50-250x faster than traditional matrix operations.

2.12 bdDiag_divide_hdf5

Performs optimized diagonal division between two datasets stored in HDF5 format. Automatically detects whether inputs are matrices (extracts diagonals) or vectors (direct operation) and uses the most efficient approach. This function is ~50-250x faster than traditional matrix operations for diagonal computations.

2.13 bdDiag_scalar_hdf5

Performs optimized scalar operations on diagonal elements of matrices or vectors stored in HDF5 format. Automatically detects whether input is a matrix (extracts diagonal) or vector (direct operation) and applies the specified scalar operation.

2.14 bdEigen_hdf5

Computes the eigenvalue decomposition of a large matrix stored in an HDF5 file using the Spectra library. This provides consistent results with the RSpectra package and can handle both symmetric and non-symmetric matrices.

2.15 bdInvCholesky_hdf5

Computes the inverse of a symmetric positive-definite matrix stored in an HDF5 file using the Cholesky decomposition method. This approach is more efficient and numerically stable than general matrix inversion methods for symmetric positive-definite matrices.

2.16 bdgetDiagonal_hdf5

Retrieves the diagonal elements from a matrix stored in an HDF5 file.

2.17 bdWriteDiagonal_hdf5

Updates the diagonal elements of a matrix stored in an HDF5 file.

2.18 bdpseudoinv_hdf5

Computes the Moore-Penrose pseudoinverse of a matrix stored in HDF5 format. The implementation is designed for large matrices, using block-based processing and efficient I/O operations.

2.19 bdWriteOppsiteTriangularMatrix_hdf5

Creates a symmetric matrix by mirroring values from one triangular part to the other in an HDF5-stored matrix. This function modifies the matrix in-place, either copying the upper triangular values to the lower triangular part or vice versa.