Error loading MPS from file

Question

Hello Miles and ITensor team!

Thanks for the great library for handling tensor network algorithms! I am using the lastest version of ITensor v3 in C++, Ubuntu 18.04 and using gcc. I made two programs, one that calculates the ground state of the Extended Hubbard Model and saves the variables sites (Hubbard object) and mps to the disk. The other program loads these objects to make another calculations. But, when I am loading these objects in the second program, I get and error in the execution that follows:

"terminate called after throwing an instance of 'std::length_error'
what(): vector::_M_default_append"

I know the error is in the following line:

MPS psi = readFromFile<MPS>(format("./data/obc/%s/N_%03d/psi",foldername,N)+append, sites);

At the moment, I was not able to reproduce the error without needing to use the specific mps that I have saved to disk, and the saved mps files may have been generated in a previous version of itensor. While I am trying to analyze in debug mode and create a code that reproduces the error, I would like some opinion about what it could be, since I don't know the conditions that this error happens, and ask two questions: (1) can previous versions of mps still be loaded in the most recent version of ITensor? (2) Is the old Hubbard siteset completely equivalent to Electron siteset (just a change in the name of the spinfull fermion sites object)?

The important parameters of my code are maxdim = 2000 (and the mps reaches this limit), and cutoff = 1E-20. Using a number of N = 20 sites this error occurs, but for N = 22 it no longer occurs.

I really appreciate any help and I will update this post as soon as I discover the reason for the error. Thanks.

Answer 1

Hi, thanks for the question. The way the write-to-disk system works in the C++ version, currently, results in us being only able to guarantee that files are readable within the same version of the code and on the same machine (due to the use of a non-portable binary format). So it's not really intended for long-term or portable storage of results, so much as for techniques like write-to-disk within DMRG to prevent running out of RAM.

Because of these limitations, we're working on HDF5 support and it's pretty far along but not quite done yet. (Another reason for the HDF5 support is so files can be read and written between the C++ and Julia versions of ITensor.)

More specifically to your issue: were the files you were reading definitely written with a different version of the library? This would be especially relevant if the older version was version 2.x and the newer one version 3.x. I couldn't tell from your question if that was definitely the case, but it would be an important detail to know. My best guess at the moment is that this was the case, and that a change we made to the internal layout of a data type from one version to another made those older files not readable by the latest version.

Finally, yes the Electron site set and Hubbard site sets are identical in terms of their physical meaning and purpose, as well as which operators they offer and QN's they support etc. But one crucial (though small) difference is that the Index tags generated by the Electron site set include the tag "Electron". (Actually I think the change from Hubbard to Electron coincided with the introduction of Index tags so there weren't any tags per se associated to the Hubbard site set.)

Best regards,
Miles

Comments

Thanks Miles!
Short story, the problem was that I was using this version when I generated the MPS via DMRG:

fd96a25 (sep 4 2019 version)
https://github.com/ITensor/ITensor/tree/fd96a25c7c74b5ad8eb0dfd38f83e42a19dabf75

and this version when I did the load to calculate observables:

7af4030 (sep 2 2020 version)
https://github.com/ITensor/ITensor/tree/7af40301470707408a62681a1ea7ba38521b6cb6

both v3. I think I assumed that the data would be saved in an equivalent way within the same v3 version, but I may have switched computers with the data, I'm still investigating. Downgrading the itensor solved the problem, and I can use the old version to get my results. I did this because I generated a large amount of data, which took a long time, and I don't know if it would be possible to re-save the MPS files in the new format (or would it be?).

Great to hear about HDF5 support, I think it would be great. Thanks again!

---

Hi, it's theoretically possible to upgrade the binary storage formats used by different versions of ITensor, but that system really wasn't designed for it. The design goal of that system was for storing temporary files to offload memory during larger calculations. So the inclusion of HDF5 is aimed more toward a portable, longer-term solution. It's almost done but we have not had much time recently to finish it, unfortunately.

Best,
Miles