Efficient way to manually increase bond dimension of MPS or IQMPS

Question

Hi,

I am looking for an efficient way to increase bond dimension of MPS or IQMPS to a given value (added values are set to zero), which is required for 1-site algorithms.

Right now, for MPS, I am using delta() to replace the link indices with indices with higher dimensions, which may not be an efficient way to do this.

Also for IQMPS, I do not have any idea how to do this, as there exist different QN sector of the existing link iqindex, and to replace it with delta(), I have to define another iqindex with same QN structure but with an extra QN() sector whose dimension will be decided depending on the final bond dimension. I have no idea, how to achieve this in a general setting.

Thanking you,
Titas

Comments

Update:
Using ``plussers()`` like function in ``mpsalgs.cc``, I am able to increase bond dimension of the IQMPS, while ``delta()`` works fine for MPS, which is also used in MPS overload of ``plussers()``.

But my problem is now conceptual. In case of IQMPS, I was adding new zero-valued IQTensor having only QN=0 sector to increase the bond dimension to a desired value. But this does not increase the number of zero-flux blocks always, and that is why after doing SVD, I am getting much reduced bond dimension (even with no truncation). Can you please suggest me, what should be the QN structure of the zero-valued IQTensor that I am adding to the original one? Because of this problem, I am stuck at hybrid algorithms for IQTensor, where 2-site algorithms are used first upto the desired bond dimension, then 1-site algorithms. In case of MPS, there is no problem.

---

I was just about to look into this problem, and suggest using `plussers()`. Glad you came across it yourself! Unfortunately right now ITensor doesn't have an easy built-in way to do this operation, but Miles and I were discussing functions we could add to help with this.

My quick answer is that you should be able to determine the QN structure from the QN structure of the IQTensor you are using to "pad" your MPS tensors (I don't think the procedure works if you only pad all of your MPS tensors with zeros, otherwise effectively the bond dimension is not being increased).

For example, using this paper as a reference:
https://arxiv.org/pdf/1501.05504.pdf
you can see in Equations (14) and (15) that they pad one tensor with zeros, but the other tensor with a tensor `P`, which in that paper they determine using Equation (19).

Then, the expanded index has a QN structure that is the direct sum of the old index and the index from `P`.

I will look into this to see if there are functions that I can add to ITensor to make this procedure easier. In theory, we could add a function like `directSum(Index i, Index j) -> Index k` where `k` is the direct sum of indices `i` and `j`. We can also add a function like `directSum(ITensor A, ITensor B) -> ITensor C`, where ITensor `C` has indices that are common between ITensors `A` and `B`, and direct sums are performed over the indices that are not common between `A` and `B`. I will try to write a prototype within the next few days that I can send to you, but please let me know if the suggestion above helps you with your own implementation.

Cheers,
Matt

---

Hi,

Please take a look at the gist: https://gist.github.com/mtfishman/1804a3a473112cb7cb80722eeac47a8b for an implementation of a `directSum` function and example code to run it. Likely you have already implemented some version of this function yourself.

I think this is a reasonable functionality to add to the ITensor library, but we would want to refine the interface before it is officially included.

Cheers,
Matt

---

Hi Matt,

Many thanks for the reply. Using ``plussers()``, now I can direct sum ITensors and IQTensors both. But I am not able to determine what should be the QN structure of the padding zero-valued IQTensor (for a given padding tensor and a mps, QN structure of the resulting tensor is not hard, which can be done using functions similar to ``plussers()`` and ``addAssumeOrth()``). This type of padding is required for original 1-site DMRG, and 1-site TDVP algorithm to increase the bond dimension of the initial product IQMPS (upto my understanding, I may be ignorant of other options).

In case of generic MPS, padding with zero ITensors works perfectly in the algorithms as long as the "Truncate" flag is set to false in SVD and as long as I don't call ``.orthogonalize()`` after padding. In this case, bond dimension is increased as desired.

In case of MPS/IQMPS, ``addAssumeOrth`` can serve the purpose of direct sum, where only one index is uncommon. But in a general setting, such function will be greatly helpful.  I just saw that you added a function in github. Thank you very much for keeping up the good work. I am very much grateful to all the contributors and authors of the library for the amazing work.

With regards,
Titas

---

Determining the QN structure of the new IQTensor depends on the method that you use for increasing the bond dimension. In general, you could pad with only zeros, but as far as I know that doesn't work in general, and in practice more sophisticated methods must be used. One method I know of is from this paper:
https://arxiv.org/pdf/1501.05504.pdf
In that case, the new QN structure is given to you by how you calculate the tensor `P` in Equation (19).

---

Conceptually, the problem you are facing with trying to pad with zeros is an indication that it is not a good method to use: padding with zeros is arbitrary, and you should pick a method (like the one in that reference) where the expanded subspace is determined in some way by the physics of your problem (i.e. by applying some form of your projected Hamiltonian to your current MPS).

---

Hi Matt,

Thanks for the reply. I will try to pad with the method given in the paper. Hopefully this will rectify my problem.

Thanks again for the help,
Regards,
Titas

---

The method of subspace expansion described in https://arxiv.org/pdf/1501.05504.pdf will probably not work in pure 1-site TDVP algorithm, where an initial (usually) product  MPS (or MPS with lower bond dimension) is written with larger bond dimension in an ad-hoc manner by setting extra terms to zero, so that every property of the original MPS remains exactly same.

---

It sounds like this recent review of time evolution methods of MPS says that the best strategy would be to use a 2-site TDVP until the bond dimension saturates at your desired bond dimension, then switch to a 1-site TDVP:
https://arxiv.org/pdf/1901.05824.pdf

In that case, the 2-site TDVP would perform the subspace expansion automatically, without having to do it manually.

---

Yes. Right now I am doing the same for both DMRG and TDVP. However, I wanted a pure 1-site TDVP for comparison. In case of generic ITensor and MPS, pure 1-site TDVP is working with padding by zero valued ITensors that I mentioned earlier.

Interestingly, this hybrid scheme (2-site version upto a certain bond dimension and then 1-site version without subspace expansion or density matrix perturbation) also works better for DMRG, especially for finding excited states (atleast for the systems that I am studying). The Hybrid scheme converges faster and more accurately at lower bond dimension, than pure 2-site DMRG. May be this is already known to the community, but is very new to me.

---

Conceptually, without any other input about the physics of the problem, the only way I can think to pick the QNs of the expanded subspace is to pick them randomly (perhaps some gaussian distribution of QNs). An alternative way could be the following, since you are doing a comparison calculation anyway:

1. Do a hybrid 2-site and 1-site TDVP calculation.
2. Then do a purely 1-site TDVP, but get the QNs for the subspace expansion from the hybrid calculation done in 1. that you will compare to.

Then, the calculations are being performed in the same MPS manifold, so it seems like the most fair comparison.

I have indeed heard that about DMRG, but had not tried it myself. It would be good for ITensor to support both 1-site and 2-site DMRG calculations, and we are also interested in supporting TDVP as soon as possible.

Cheers,
Matt

---

Yes. Your idea about getting QNs from the hybrid scheme is very promising. I should try that.

Thanks again,
Titas

Answer 1

Just to give an official answer to summarize the comments above:

1. As a technical problem of expanding the subspace of an IQMPS, it is not directly built into ITensor right now, though with some modification of existing code in itensor/mps/mpsalgs.cc it is not hard to do. For example, take a look at: https://gist.github.com/mtfishman/1804a3a473112cb7cb80722eeac47a8b which shows an implementation of a "partial direct sum" operation of two IQTensors, which should be sufficient for doing a subspace expansion of an IQMPS. This is something that we will try to support officially in ITensor as soon as possible.

2. A general question which is application dependent is how to choose the QNs of an expanded IQMPS subspace. The answer is that, when possible, it is best to determine the expanded subspace using the physics of your problem. For a single site DMRG calculation, a popular method seems to be ( https://arxiv.org/pdf/1501.05504.pdf ), but various methods have been proposed. Alternatively, a method that would work fairly generically would be to first perform a 2-site version of the MPS algorithm you are interested in (for example, 2-site DMRG or 2-site TDVP), which will automatically expand the subspace. Then, you can switch to a 1-site version once the desired bond dimension is reached.