Memory issue with DMRG

Question

Hi Miles,

I was trying to run DMRG on clusters, but the code got killed after a couple of sweeps (with max bond dimension already reached), not finishing the number of sweeps set. What I would assume was that if one particular sweep with max bond dimension could run successfully without memory issue, then the rest of the sweeps should be able to run without any memory issue, because they are supposed to be almost identical processes.

Maybe I'm wrong, but I would appreciate if you could some insight about this.

Thanks a lot,
-Mason

Answer 1

Hi Mason,
Thanks for the question: I've run into a similar behavior before, and I believe it's to do with something fundamental about DMRG. I think what's going on is that for first sweep where the max bond dimension is the largest one you set, or the one you are hoping will continue to fit in memory for subsequent sweeps, actually many of the MPS link or bond indices have not yet reached their maximum value. So even though they are allowed to reach this value, they actually haven't yet. Relatedly, tensors such as the "projections" of the Hamiltonian into pieces of the MPS have indices which are not yet as large as they could be.

Then, in the next sweep, finally all indices start to reach their maximum values and this is when you run out of memory.

Does that sound plausible? At any rate, the key point is that you are running out of memory somewhere even if it's not where you were initially expecting. So then the main thing you can do is to start having DMRG store tensors to disk before that point.

Best regards,
Miles

Comments

yes, that makes a lot of sense to me. Thank you Miles!