Zip-up compression for FiniteMPO-FiniteMPS product - #470
Conversation
|
Please let me know if there are actually better functions in MPSKit that does more or less the same thing. |
|
@lkdvos I cannot run the format check on GitHub either. So there's something to be fixed. |
|
Thanks for getting this started! I'll look into the formatter, let's not worry about that here. For consistency, it would probably make the most sense to match this into I'm a bit confused about the necessity for As an alternative, I would suggest writing this in terms of primitive pieces that can be overloaded for the PEPS case, which keep the PEPS layers separate to keep the computational benefits in. Here that would probably just be |
Codecov Report❌ Patch coverage is
... and 80 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Ah, the same interface debate as
What is meant by "exterior vectors"?
Indeed I realized this later. We just overload |
Here I mean that a
I think this is probably sensible, it might additionally store something like a direction or a backend and allocator in the future. |
|
Initializing with undef just for matching interface is a bit unnecessary for me. What about also calling the function |
|
It's just really annoying to have both |
|
Thanks for putting this together — the zip-up contraction is a nice addition. A few questions/comments while reading through it, cross-checking against the original references: 1. Naming The algorithm is implemented here as 2. Missing truncation-only sweep As I understand the original algorithm, it consists of two sweeps: one sweep that does the contraction with only a loose/minimal truncation (to keep the intermediate bond dimension under control without discarding meaningful weight), followed by a second sweep that performs the actual truncation down to the target Related to this: Paeckel et al. (Annals of Physics 411, 167998 (2019), DOI: 10.1016/j.aop.2019.167998 give a concrete prescription for the intermediate/minimal truncation stage — using a looser bond dimension (2× the target 3. Orthogonality center of the MPO Stoudenmire & White also move the orthogonality center of the Hamiltonian/MPO to the last site (given this is done right-to-left here). I don't see that happening in this implementation — is the MPO's orthogonality center supposed to be moved before by the user, or is this something that still needs to be added? Happy to be corrected on any of this if I'm misreading the code or the references — just want to make sure the implementation matches the papers it's based on before it lands. |
|
@VinceNeede Hope the following answers your questions.
From you description, I think this is done by
By construction the |
|
Well, after some reading, the "zip-up" algorithm and the |
|
Thanks for the updates, glad the naming got sorted! On point 2: You're right that On point 3 (orthogonality center): To clarify, I meant the gauge of the MPO, not the resulting MPS. In the original algorithm, this gauge-fixing step isn't primarily about canonical form for its own sake — it's there to control conditioning. Quoting Stoudenmire & White directly: the arrangement guarantees that the basis to the right of the current site, from the product of the MPO and the MPS, is "not drastically ill-conditioned," meaning no basis state ends up with a norm much bigger than one. I don't think the Jordan-block structure of a Hamiltonian MPO gets you this for free. The triangular structure only constrains the flow of information between channels (no cycling back to an earlier block) — it says nothing about the magnitude of the entries. For short-range Hamiltonians with O(1) couplings, the channels are probably all reasonably scaled, so this may not show up in practice. But for long-range or multi-scale Hamiltonians (e.g. power-law couplings, or terms spanning very different energy scales), the "already-closed" channels can accumulate sums of very different magnitude as the sweep progresses — exactly the kind of basis ill-conditioning the gauge-fixing step is meant to prevent. So I think this is a real, if perhaps rare in typical test cases, gap rather than something the Jordan form makes moot. That said, I don't know whether MPSKit has any existing notion of a gauge/orthogonality center for |
|
I would refrain from incorporating SvdCut changebonds into the zip-up algorithm to keep things focused. Even in the original paper the second step is referred to as another "fitting algorithm". For gauging MPO, I guess you can always convert it to an MPS with 2 physical legs per site. FiniteMPO appears to be just a thin wrapper over the vector of MPO tensors, so it does not have the complete machinery to handle the gauges like FiniteMPS. |
|
One more clarification on point 2, since I think we might be talking about two different things: the "fitting algorithm" the paper refers to (e.g. a DMRG-style variational sweep) is a separate, optional refinement that operates once you're already at the target bond dimension — it improves the overlap with the exact result but doesn't change the bond dimension itself. The two-sweep structure I meant is different: it's how zip-up itself reaches the target bond dimension without losing information along the way. The intermediate sweep has to use a loose truncation precisely because it isn't reaching the target bond dimension yet — cutting straight to the target on a single pass risks discarding weight that the second sweep would have recovered. The second sweep is the one actually responsible for reaching the desired final bond dimension, using the better-conditioned information available after the first pass. So this two-sweep step is intrinsic to the zip-up compression itself, and only after it's done would a separate fitting algorithm optionally be applied. Just wanted to make that distinction explicit, since I think that's where the disagreement comes from. On the MPO point: given |
|
What would you suggest about the interface if we do two successive truncations? Do we pass two |
|
For the interface, I'd suggest two separate keyword arguments, something like
This also matches what ITensor (maintained by Stoudenmire and White themselves) does in practice — their |
|
ITensor reference is very helpful! I'll soon update to match the behavior there. Not all truncation strategies have a dimension or error cutoff. So for automatic determination of |
|
@VinceNeede An explicit constructor |
|
Thanks for adding the two-sweeps mechanism! I agree that this constructor differs from the interface used by the other algorithms. That said, introspecting One correction on the truncation-scheme side: the cutoff as intended by Miles and White is the sum of squares of the discarded singular values, relative to the total sum of squares (see ITensor docs). This corresponds to |
VinceNeede
left a comment
There was a problem hiding this comment.
Hi, since I was browsing the PR I had a couple of questions, feel free to disregard if there's context I'm missing!
| @@ -53,12 +68,14 @@ function approximate((O, ψ)::Tuple{Any, <:FiniteMPS}, alg::Zipup) | |||
| if i == N | |||
| return Aᶻ | |||
| else | |||
| AL, C, _ = left_gauge(Aᶻ, alg.alg_gauge) | |||
| AL, C, _ = left_gauge(Aᶻ, alg_zipup) | |||
| carry = C | |||
| Fₗ = Fᵣ | |||
| return AL | |||
| end | |||
| end | |||
|
|
|||
| return FiniteMPS(As; normalize = false, overwrite = true) | |||
| ψ′ = FiniteMPS(As; normalize = false, overwrite = true) | |||
There was a problem hiding this comment.
I was wondering if this could be made into an in-place function — I was implementing something similar for TEBD recently, so thought I'd flag the pattern here too, in case it's useful.
| Fᵣ = fuser(A, left_virtualspace(ψ, 1), left_virtualspace(mpo, 1)) | |
| C_prev = nothing | |
| for site in 1:N | |
| A1 = site == start ? ψ.AC[site] : ψ.AR[site] | |
| Fₗ = Fᵣ | |
| Fᵣ = fuser(A, right_virtualspace(ψ, site), right_virtualspace(mpo, site)) | |
| fused = _fuse_mpo_mps(mpo[site], A1, Fₗ, Fᵣ) | |
| actual = isnothing(C_prev) ? fused : _mul_front(C_prev, fused) | |
| if site == N | |
| ψ.AC[site] = actual | |
| else | |
| AL, C, = left_gauge(actual, alg_zipup) | |
| ψ.AC[site] = (AL, C) | |
| C_prev = C | |
| end | |
| end |
There was a problem hiding this comment.
I'm definitely not against this, but I'm not actually sure this will work, since it does leave the MPS in an invalid intermediate state between the iterations. (AL, C) does not actually connect to the next site until the you fuse the next site as well. I'm wondering if the automated gauging machinery might not at some point get you into trouble here, so it might not be worth it
| @@ -53,12 +68,14 @@ function approximate((O, ψ)::Tuple{Any, <:FiniteMPS}, alg::Zipup) | |||
| if i == N | |||
| return Aᶻ | |||
| else | |||
| AL, C, _ = left_gauge(Aᶻ, alg.alg_gauge) | |||
| AL, C, _ = left_gauge(Aᶻ, alg_zipup) | |||
| carry = C | |||
| Fₗ = Fᵣ | |||
| return AL | |||
| end | |||
| end | |||
|
|
|||
| return FiniteMPS(As; normalize = false, overwrite = true) | |||
| ψ′ = FiniteMPS(As; normalize = false, overwrite = true) | |||
There was a problem hiding this comment.
I'm definitely not against this, but I'm not actually sure this will work, since it does leave the MPS in an invalid intermediate state between the iterations. (AL, C) does not actually connect to the next site until the you fuse the next site as well. I'm wondering if the automated gauging machinery might not at some point get you into trouble here, so it might not be worth it
`physicalspace` of an MPO tensor returns its output space, which is the wrong one to compare against a state that the MPO is applied to whenever the input and output physical spaces differ. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`_fuse_mpo_mps_left` and `_fuse_mpo_mps_right` leave the virtual legs on one side of the local MPO-MPS product unfused, such that a subsequent factorization across that side supplies the fuser of the next site. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`approximate!(ψ, (O, ϕ), alg::Zipup)` now writes the product into `ψ`, which is a destination rather than an initial guess and may alias `ϕ`; the out-of-place method allocates it as `similar(ϕ, T)` with the promoted scalar type. Both return `(ψ, ϵ)`, reporting the truncation error that was previously discarded. The zip-up sweep no longer fuses the right virtual bond of every site: after multiplying with the left fuser, the right factor of the truncated decomposition is simultaneously the truncation carry and the fuser of the next site, so the object with both bonds fused is never constructed. The sweep is split into in-place `zipup!` and `zipdown!` helpers, and all compatibility checks happen in the `approximate!` entry point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`zipup!` and `zipdown!` are merged into `zip_left_right!`, which zips up from left to right and then truncates from right to left, and its mirror image `zip_right_left!`. The new `left_to_right` field of `Zipup` selects between them. The right-to-left sweep needs the mirrored partially fused contraction, whose local tensors are already partitioned across the new bond, so the leg permutation that `right_gauge` applies to MPS tensors has to be skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both sweep directions, with and without the zip-down pass, are checked against the dense MPO-MPS product, along with an empty destination, a destination with unrelated contents, a destination aliasing the input, scalar type promotion, non-trivial boundary virtual spaces, and an MPO whose input and output physical spaces differ. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The zip-up sweeps are `approximate` algorithms, so their tests belong with the other ones rather than in a file of their own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lkdvos
left a comment
There was a problem hiding this comment.
I've added some of the things I had in mind, including the option to alter the sweep direction if this ever shows up. I did some small benchmarks and it is definitely quite a bit faster like this, I hope the code is still readable and can be used appropriately.
If the tests turn green, for me this is good to go, but I'll definitely wait for your call.
As always, thanks for the contribution, much appreciated!
|
OK to merge for me, but one test timed out. |
|
The macos tests often time out, this is unrelated to this PR |
Both landed after these pages were drafted, and the pages enumerate the time-evolution options, so leaving them out would have shipped a listing that is wrong by omission. `BUG` (#468) gets a how-to recipe, a paragraph in the algorithm landscape, and a row in the decision table. Two things are worth stating explicitly and are easy to get wrong from the name alone: it is the one single-site integrator that can change bond dimension, and a `truncrank(D)` leaves the state at `2D`, because each half-sweep only truncates the previous one's basis augmentation — so a follow-up `SvdCut` is needed to actually land on `D`. `Zipup` (#470) is documented alongside the propagator-MPO route it belongs to, including the two API differences from the variational path: no initial guess, and a 2-tuple return. It is not mentioned on the statistical-mechanics page, since it is finite open-boundary only. `approximate`/`approximate!` had no topical reference home at all; they now live on the time-evolution reference page next to `Zipup`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both landed after these pages were drafted, and the pages enumerate the time-evolution options, so leaving them out would have shipped a listing that is wrong by omission. `BUG` (#468) gets a how-to recipe, a paragraph in the algorithm landscape, and a row in the decision table. Two things are worth stating explicitly and are easy to get wrong from the name alone: it is the one single-site integrator that can change bond dimension, and a `truncrank(D)` leaves the state at `2D`, because each half-sweep only truncates the previous one's basis augmentation — so a follow-up `SvdCut` is needed to actually land on `D`. `Zipup` (#470) is documented alongside the propagator-MPO route it belongs to, including the two API differences from the variational path: no initial guess, and a 2-tuple return. It is not mentioned on the statistical-mechanics page, since it is finite open-boundary only. `approximate`/`approximate!` had no topical reference home at all; they now live on the time-evolution reference page next to `Zipup`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This PR adds
approximate((O, ϕ), ::Zipup)for open-boundaryO::FiniteMPOacting onϕ::FiniteMPS, without normalization, first introduced in 1002.1305:The in-place version merely uses
ψas the output destination, instead of initialization. This provides a streaming MPO-MPS application plus SVD compression path. The algorithm structZipupcontrols the SVD algorithm and truncation strategy.Motivation
This is useful for PEPS boundary-MPS style contractions (QuantumKitHub/PEPSKit.jl#396), where applying a double-layer transfer MPO to a boundary MPS can temporarily create very large virtual bonds. The existing initialization pattern,
first materializes the full enlarged
O * ϕacross all sites, then compresses it. That is unnecessarily memory-heavy when the product is only needed as an initialization that will be further optimized withapproximate.The new
Zipupalgorithm contracts one MPO/MPS site pair at a time from right to left, absorbs the "carry" (residue tensor produced by right-orth gauge conversion) from the already-compressed right block, and immediately truncates before moving left. This avoids storing the fully enlarged product on every site.Notes
The
Zipupapproximation matcheschangebonds(O * ϕ, SvdCut(; trscheme); normalize=false)when no meaningful singular values are discarded, but can differ under real truncation becauseZipuptruncates while building the product rather than after materializing the full product.