SVD operation is too slow.

Question

Hello.

I am studying the tensor renormalzation group.
In the renormalization step, I use SVD to obtain a truncated matrices.

 auto A = replaceTags(replaceTags(R_L,"x2","x1"),"x4","x3")*R_R;
 auto [U, S, V] = svd(A, {ind_RL},{"MaxDim",dxn,"LeftTags","ulink","RightTags","vlink"});

Problem is, when I run the code with a 6400 x 6400 matrix A, SVD takes about 1000 seconds.

When I run SVD with mkl LAPACKE_zgesvd with the same matrix, it takes about 75 seconds.
( This process includes storing the tensor A to 1D array and assigning the output elements of [U, S, V] to the ITensor container after the SVD operation.)

I am wondering why this happened. I thought running ITensor SVD and mkl LAPACK routine should take the same time...
Thanks.

Best,
hexagon

Answer 1

Good question, thanks. By default, ITensor uses a custom SVD implementation which is more reliable in general than the LAPACK implementations, which have occasionally crashed for various users. So until this is fixed we offer our custom SVD as the default.

But recently we added an option to call the "gesdd" and "gesvd" LAPACK methods which can be much faster (and more accurate) than ours.

So I would recommend you add "SVDMethod=","gesdd" to the named arguments you are passing to the svd function, and you should see much better performance. For this to work you will need to pull and compile the latest version of ITensor (v3 branch) if you haven't already.

Please let me know if you don't see a speedup or comparable performance to your other code which uses LAPACK!

Best regards,
Miles

P.S. I just updated the documentation to list the SVDMethod option: http://itensor.org/docs.cgi?vers=cppv3&page=classes/decomp

Comments

Hi Miles,

I just saw this post. May I have some further questions:
1. If I give {"SVDMethod=","gesdd"} argument into DMRG function, would it also use SVD in LAPACK in DMRG?
2. In the case that LAPACK SVD fails, would the program just crashes (which is preferred), or it would just returns some wrong results?

Thanks!
Chia-Min

---

Hi Chia-Min,
I'm replying here for the benefit of others, even though we discussed this by email too. This is a good question, as the behavior of ITensor DMRG is a bit complicated in this regard. Sometimes ITensor DMRG doesn't use an SVD to factorize the wavefunction because for a lower-accuracy (higher cutoff) calculation it can use a density matrix decomposition, which can be faster.

So only when DMRG uses an SVD will the {"SVDMethod=","gesdd"} argument have an effect. To force DMRG to always use an SVD with the LAPACK gesdd routine, you can pass the following pair of named arguments: {"UseSVD=",true,"SVDMethod=","gesdd"}. To see that it is working, check the speed of your DMRG calculations before and after passing this argument. (If there's no speed difference, it could be just because you are doing a small-bond-dimension calculation, as the speed difference for gesdd only becomes significant for matrices with linear dimension about 1000.)

If the LAPACK SVD fails, you'll most likely just get a crash.

Thanks for the question,
Miles