estimateOptimalBlockSize

C++ Function Reference

1 Signature

hsize_t BigDataStatMeth::estimateOptimalBlockSize(hsize_t matrix_size, double available_memory_mb=100.0)

2 Description

Utility function to estimate optimal block size for diagonal matrix operations.

3 Parameters

matrix_size (hsize_t): Size of the matrix
available_memory_mb (double): Available memory per thread in MB (default: 100MB)

4 Returns

Optimal block size for diagonal processing

5 Details

Calculates optimal block size based on available memory and matrix size. This function is specifically optimized for diagonal-only processing where only diagonal blocks are read/written, not the entire matrix.

6 Source Code

Implementation

File: inst/include/hdf5Utilities/hdf5Diagonal.hpp • Lines 851-863

inline hsize_t estimateOptimalBlockSize(hsize_t matrix_size, double available_memory_mb = 100.0) {
        // Each diagonal block uses block_size^2 * 8 bytes (double precision)
        // Keep blocks under specified memory limit per thread
        double bytes_per_mb = 1024.0 * 1024.0;
        double max_elements = (available_memory_mb * bytes_per_mb) / sizeof(double);
        hsize_t max_block_size = static_cast<hsize_t>(std::sqrt(max_elements));
        
        // Ensure block size is reasonable for diagonal processing
        hsize_t min_block = 64;   // Minimum for efficient I/O
        hsize_t max_block = std::min(static_cast<hsize_t>(4096), matrix_size); // Maximum practical size
        
        return std::max(min_block, std::min(max_block_size, max_block));
    }

7 Usage Example

#include "BigDataStatMeth.hpp"

// Example usage
auto result = estimateOptimalBlockSize(...);