Minor edits to Section 3 - #10
Open
anyzelman wants to merge 1 commit into
Open
Conversation
hartwiganzt
reviewed
Aug 20, 2026
| By adopting specialized data structures that omit the majority of (or all) zeros, we can unlock significant savings in memory footprint, memory access volume, and computational cost. Rather than wasting cycles and bandwidth on arithmetic operations with zeros and reading zeros from RAM, sparse data structures explicitly store only nonzero entries (and maybe some explicit zeros), implying zeros for all non-explicitly stored values. Using sparse data structures usually increases the complexity in accessing nonzero matrix entries, and each sparse data format necessitates a tailored ecosystem of routines that can operate on that specific format. | ||
|
|
||
| Over the decades, a set of commonly used sparse matrix formats has emerged, providing a versatile set of data structures that allow for productivity and performance. The formats we identified as most common are \textit{Compressed Sparse Row} (CSR)\footnote{Also known as Compressed Row Storage (CRS)}, \textit{Compressed Sparse Column} (CSC), and \textit{Coordinate} (COO). | ||
| Over the decades, a set of commonly used sparse matrix formats has emerged, providing a versatile set of data structures that allow for productivity and performance. The formats we identified as most common are \textit{Compressed Sparse Row} (CSR)\footnote{Also known as Compressed Row Storage (CRS) with its column-major variant also known as CCS.}, \textit{Compressed Sparse Column} (CSC), and \textit{Coordinate} (COO). |
Contributor
There was a problem hiding this comment.
I do not understand: what is column-major CSR? Is that CSC?
Contributor
Author
There was a problem hiding this comment.
Yes, Compressed Sparse Columns and Compressed Column Storage are the same, just as CRS and CSR are the same. The naming divergence stems from the textbooks of Saad and Bai et al., although that's more for historians. I kept it here as one footnote, but up to you-- I feel half of the world uses CRS, the other CSR (with no-one using Yale;)
(I once went deep in Leuven's library and the earliest CRS/CSR paper I found was actually from Yale's sparse matrix package; Eistenstat et al., '77, who defined it rather than referring to it-- meaning they might have been first to discover it)
hartwiganzt
reviewed
Aug 20, 2026
| \end{itemize} | ||
|
|
||
| The third archetype, the \textbf{Sparse Triangular Solve} (SpTRSV), solves a linear system $Ax=b$ with sparse triangular matrix $A$. The sparse triangular matrix $A$ may be stored as a triangular matrix, or as a full matrix where the SpTRSV has to extract the triangular part on-the-fly during kernel execution. For enabling parallelism, this archetype, as opposed to the other archetypes, requires the SpTRSV to heavily rely on auxiliary information extracted from an analysis of the sparsity pattern. As the operation is inherently sequential: each element of the output vector may depend on previously computed values. To extract parallelism, the implementations typically construct a dependency graph or utilize level-scheduling to identify which components can be computed simultaneously, making performance highly sensitive to the specific sparsity pattern of the matrix. This analysis can often benefit from user hints provided in the property tags of the sparse matrix (detailed in Section~\Cref{sec:views}). | ||
| The third archetype, the \textbf{Sparse Triangular Solve} (SpTRSV), solves a linear system $Ax=b$ with sparse triangular matrix $A$. The sparse triangular matrix $A$ may be stored as a triangular matrix, or as a full matrix where the SpTRSV has to extract the triangular part on-the-fly during kernel execution. For enabling parallelism, this archetype, as opposed to the other archetypes, requires the SpTRSV to heavily rely on auxiliary information extracted from an analysis of the sparsity pattern-- otherwise, it must execute sequentially as each element of the output vector may depend on all previously computed values. To extract parallelism, the implementations typically construct a dependency graph or utilize level-scheduling to identify which components can be computed simultaneously, making performance highly sensitive to the specific sparsity pattern of the matrix. This analysis can often benefit from user hints provided in the property tags of the sparse matrix (detailed in Section~\Cref{sec:views}). |
hartwiganzt
reviewed
Aug 20, 2026
| \item Elementwise Multiplication (sparse matrix .* sparse matrix -> sparse matrix) | ||
| \item Format conversions: sparse matrix -> sparse matrix) | ||
| \item Transposition: sparse matrix -> sparse matrix | ||
| \item SpGEMM ( sparse matrix * sparse matrix -> sparse matrix), |
hartwiganzt
reviewed
Aug 20, 2026
|
|
||
|
|
||
| In contrast to the SpMV kernel, the \textbf{Sparse Matrix-Matrix Multiplication} archetype (SpGEMM, computing the product of two sparse matrices) represents a class of much more complex routines that share the property of generating a sparse matrix as output with unknown sparsity pattern. In consequence, the sparse data structure for the output can not be allocated prior to the invocation of the functionality. The functionality can only be implemented as a multi-stage routine. The process begins with the first stage computing the size of the output, the allocation of the output data structure, and the final stage filling the output data structure. In~\Cref{sec:ownership} we will discuss the ownership of the input and output data structures, but already here we mention that the sparse input matrices could be enhanced with optimization-enabling information, the sparse output matrix is expected to be a plain sparse matrix. For the SpGEMM routine, different parallelization strategies can be realized, ranging from row-parallel and column-parallel to sub-matrix parallel and element-parallel, but all coming with their strategy-specific downsides. Functionality falling into the group represented by the SpGEMM include | ||
| In contrast to the SpMV kernel, the \textbf{Sparse Matrix-Matrix Multiplication} archetype (SpGEMM, computing the product of two sparse matrices) represents a class of much more complex routines that share the property of generating a sparse matrix as output with unknown sparsity pattern. In consequence, the sparse data structure for the output can not be allocated prior to the invocation of the functionality. The functionality can only be implemented as a multi-stage routine. The process begins with the first stage computing the size of the output, the allocation of the output data structure, and the final stage filling the output data structure. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
see also upcoming email