CC BY
8
2018, Т. 160, кн. 4 С. 788-796
УЧЕНЫЕ ЗАПИСКИ КАЗАНСКОГО УНИВЕРСИТЕТА. СЕРИЯ ФИЗИКО-МАТЕМАТИЧЕСКИЕ НАУКИ
ISSN 2541-7746 (Print) ISSN 2500-2198 (Online)
UDK 512.64+517.98
A LOW-RANK APPROXIMATION OF TENSORS AND THE TOPOLOGICAL GROUP STRUCTURE OF INVERTIBLE MATRICES
R.N. Gumerov, A.S. Sharafutdinov
Kazan Federal University, Kazan, 420008 Russia Abstract
By a tensor we mean an element of the tensor product of vector spaces over a field. Up to a choice of bases in factors of tensor products, every tensor may be coordinatized, i.e., represented as an array consisting of numbers. The properties of the tensor rank, which is a natural generalization of the matrix rank, have been considered in this paper. The topological group structure of invertible matrices has been studied. The multilinear matrix multiplication has been discussed from the viewpoint of transformation groups. We treat a low-rank tensor approximation in finite-dimensional tensor products. It has been shown that the problem on determining the best rank- n approximation for a tensor of size n x n x 2 has no solution. To this end, we have used an approximation by matrices with simple spectra.
Keywords: approximation by matrices with simple spectra, group action, low-rank tensor approximation, norm on tensor space, open mapping, simple spectrum of matrix, tensor rank, topological group of invertible matrices, topological transformation group
Introduction
Tensors are ubiquitous in sciences. The subject of tensors is an active research area in mathematics and its applications (see, for example, [1-3] and references therein).
This paper is devoted to the tensor rank and a low-rank approximation of tensors. The tensor rank can be considered as a measure of complexity of tensors. Therefore, one is often required to find an approximation of a given tensor by tensors with lower tensor ranks. In particular, the best low-rank approximation problem for tensors is of great interest in the statistical analysis of multiway data (see, for example, references in [4, p. 1085]). As is known, in general, the best low-rank approximation problem for tensors is ill-posed [4, 5].
A part of motivation for this work comes from our study of the complexity of tensors in homological complexes of Banach spaces [6, 7]. The main part of motivation comes from the results in [4, 8, 9] on tensors in finite-dimensional spaces. In this paper, we consider tensors in finite-dimensional spaces with the Euclidean topology. The properties of these tensors are closely related to the topological group structure of invertible matrices. Here, we deal with the natural topological group action on a space of tensors. We show the ill-posedness of the best rank- n approximation problem in the space of tensors of size n x n x 2.
1. Tensor rank and its properties
As usual, N stands for the set of all natural numbers. In the sequel, l, m, n G N and l, m, n ^ 2.
Throughout the note, F will denote either the field of complex numbers C or the field of real numbers R. For an element x G Fl, we use the notation x = (x1,... ,xi )T, where Xi G F, i = 1, .. . ,l.
We denote by Flxm, or by Mi,m(F), the linear space of all matrices A = (aij) of size l x m, where aij G F, i = 1,..., l, j = 1,... ,m. The space of all square matrices of order n over the field F is denoted by Mn(F). The general linear group of degree n, i.e., the group of invertible matrices in Mn(F), is denoted by GLn(F). The symbol En stands for the identity matrix in Mn(F).
For x = (xi,..., xl)T G Fl and y = (y1,... ,ym)T G Fm, the matrix x ® y G Flxm is given by
x ® y = (xiyj), where i = 1,... ,l, j = 1,... ,m.
Let Flxmxn be the linear space of all arrays A = (aijk) of size l x m x n, where aijk G F, i = 1,.. .,l, j = 1,... ,m, k = 1,... ,n. For a tensor A = (aijk) G Flxmxn, we also use the following notation:
A =[Ai\^-\An], where, for every r = 1,..., n, the slice Ar is defined by
Ar = (aijr) G Flxm, where i =1,...,l, j = 1,...,m.
For x = (x1,...,xi)T G Fl, y = (y1,...,ym)T G Fm, z = (z1,...,zn)T G Fn,, we define the array x ® y ® z G Flxmxn by
x ® y ® z = (xiyj Zk), where i = 1,. .. ,l, j = 1,. .., m, k = 1, .. ., n.
Let us consider the bilinear mapping 9 and the trilinear mapping t defined as follows:
9: Fl x Fm —► Flxm : (x, y) ^ x ® y;
t : Fl x Fm x Fn —> Flxmxn : (x, y, z) ^ x ® y ® z.
It is well known that the pairs (Flxm,9) and (Flxmxn,T) are the tensor products for the corresponding linear spaces. In what follows, elements of the spaces Flxm and Flxmxn are called tensors.
For the basics of algebraic tensor products, we refer the reader, for instance, to [2, Part I, Ch. 3], [10, Ch. 1], and [11, Ch. 2, § 7].
Both of the l1 -norms on the linear spaces Fmxn and Flxmxn will be denoted by the same symbol || • || 1. We recall that the value of the l1 -norm at a tensor A is defined as the sum of absolute values of all entries in A.
All norms on a space of tensors are equivalent and generate the same topology that is called the Euclidean topology. The convergence of a tensor sequence
{At} = {(aj)} C Flxmxn, t G N,
to a tensor A = (aijk) G Flxmxn with respect to this topology is exactly the entrywise convergence, i.e., for every fixed triple of indices i = 1, . . . , l, j = 1, . . . , m, k = 1, . . . , n, one has the equality
t lim ajk = aijk.
In the sequel, we consider spaces of tensors endowed with the Euclidean topologies. The general linear group GLn(F) is a topological group with respect to that topology.
' 1 0 0 -1 "
0 -1 -1 0
Definition 1. Tensors A e Flxm and B e Flxmxn are said to be elementary (or decomposable) if A = a ® b and B = x ® y ® z for some vectors a, x e Fl, b, y e Fm and z e Fn.
Definition 2. A tensor A e Flxm or a tensor B e Flxmxn has the tensor rank r if it can be written as a sum of r elementary tensors, but no fewer. We will use the notation rank(A) (or rank^(A)) for the tensor rank of A. Therefore, we may write
r
rank(B) = min {r | B = ^ Xi ® yi ® zi, where xi e Fl, yi e Fm, zi e Fn}.
i=i
As is well known, for A e Flxm, the tensor rank rank(A) is exactly the matrix rank and, for A e Rlxm, the equality rankR(A) = rankC(A) is valid. On the other hand, the tensor rank rank (A), where A e Flxmxn, depends on a field F. It is clear that the inequality rankc(A) ^ rank-R(A) holds.
Example. Let
A
It can be shown that rankR(A) = 3 and rankC(A) = 2 (see [2, Example 3.44]).
Then, we introduce the topological group, which is the Cartesian product of general linear groups
GLim,n(F) := GLl(F) x GLm(F) x GLn(F)
and consider a GLiim,n(F)-action on the space Flxmxn (see also [4, Section 2.1]). For the notions and facts in the theory of topological transformation groups, we refer the reader, for example, to [12] and [13].
Let us take elements A e Flxmxn and (L, M, N) e GL,mn(F) given as follows: A = (ajjk), L = (Xpi), M = (pqj), N = (vrk). The tensor A is transformed into the tensor B = (L, M, N) ■ A e Flxmxn by the rule:
l , m , n
B = (bpqr) e Flxmxn, where bpqr = ^ Xpi^qjVrkajjk.
i,j,k=l
Thus, we have the mapping called the multilinear matrix multiplication
$: GLlm,n(F) x Flxmxn —► Flxmxn : ((L, M, N), A) i—► (L, M, N) ■ A,
which was studied in [4, Sections 2.1, 2.2, and 2.5]).
The mapping $ is considered below from the viewpoint of transformation groups.
Proposition 1. The following properties are fulfilled:
1) the triple (GLlmnn(F), Flxmxn, $ is a topological transformation group;
2) every orbit for the GLlmhn(F) -action consists of elements of the same tensor rank;
3) the group action of GLlmn(F) on Flxmxn is non-effective;
4) the space Flxmxn is non-homogeneous under the GLl,m,n(F) -action.
Proof. 1) We show only the continuity of the multilinear matrix multiplication $. To this end, we take sequences {(Lt,Mt,Nt)} C GL,,mn(V) and {At} C Flxmxn, t e N, that converge to (L,M,N) e GLKmn(V) and A e Flxmxn, respectively. Hence, we have the coordinatewise convergence:
lim Lt = L, lim Mt = M, lim Nt = N.
t—>- + tt t—>- + tt t—>- + tt
We introduce the following notations:
Lt = (Xtpi), Mt = (ptqj), Nt = (Vrk), At = (a\jk);
L = (Api), M = (v*qj), N =(vrk), a = (aijk). Then, we have the equalities for entries of tensors:
lim Xpi = Xpi, lim ntqj = fiqj, lim vfrk = Vrk, lim atijk = aijk. t—tt r t—tt t—tt t—tt J
We define the constants as
M\ = sup |Api| , M2 = sup | , M3 = sup |^rk | , Mi = sup \ajk | .
p,i,t q,j,t r,k,t i,j,k,t
In addition, let us set (Lt, Mt, Nt) ■ At = (b\jk) and (L, M, N) ■ A = (bijk). Then, for every e > 0 there exists T e N such that for all t > T we have the following inequalities:
|bpqr bpqr\ —
l,m,n l,m,n
^pi^qj Vrkaijk — ^pi^qj Vrk aljk
<
i,j,k=1 i,j,k=1 l,m,n
— \^Pi^qj Vrk aijk — ^Pi ¡\j Vrk aijk 1 —
i}j}k=l
l,m,n
— 53 O^i — ^Pi\ \Vqj Vrk aijk\ + \^qj — ¡¡qj \ \^piVrk aijk\) +
i}j}k = l
l,m,n
+ 53 (\Vrk — Vrk \ ^pi^qjaijk \ + \aijk — aljk \ Vtk\) —
i}j}k=l
^ ( eM2M3M4 eM1M3M4 eM1M2M4 eM1M2M3
— ^ \ AlmnMnMnM* + âlmn.M-, Mn M, + AlmnM MnM* +
i,j,k=1
\4lmnM2M3MA 4lmnM1M3M4 4lmnM1M2M4 4lmnM1M2M3
l,m,n
lmn
<
Ee
lmn
i,j,k=1
Hence, the sequence &((Lt, Mt, Nt), At) converges to the tensor &((L, M, N), A), as required.
2) See the proof of Lemma 2.3(2) in [4].
3) Take (L,M,N) = (aEl, /3Em,jEn) with arbitrary scalars a,3,Y G F satisfying the condition a3Y = 1- Then, for every A = (aijk) G Flxmxn, we have
$((L, M, N),A) = (a3Yaijk) = (aijk) = &((EhEm,En),A).
This shows that the action is non-effective.
4) Consider tensors A = xi ® yi ® zi and B = xi ® yi ® zi + x2 ® y2 ® z2, where {xi, x2} C Fl, {yi, y2} C Fm and {zi, z2} C Fn are pairs of linear independent vectors. Then, one has rank (A) = 1 and rank (B) = 2. Using item 2), we obtain the desired conclusion.
We have the following proposition on the semicontinuity of the tensor rank (see [4, Proposition 4.3, Theorem 4.10]).
Proposition 2. The following properties are fulfilled:
1) for every r < min(l,m), the set Sr (l,m) := {A e Flxmlrank(A) < r} is closed;
2) there exists r such that the set Sr(l,m,n) := {B e Flxmxnlrank(B) < r} is not closed.
2. The topological group of invertible matrices and approximations of matrices and tensors
In this section, we consider a low-rank approximation of tensors in the space Cnxnx2 .
To this end, we should first formulate Bi's criterion for square-type tensors in the space Cmxmxn (see [14, Proposition 2.5]).
Proposition 3. Let A = [Ai| ... lAn] be a tensor in Cmxmxn, where n > 2, and let Ai e Cmxm be a nonsingular matrix. Then, the tensor rank of A is equal to m if and only if the matrices A2A-1, .. ., AnA-i can be diagonalized simultaneously.
Secondly, we recall some algebraic and topological definitions and facts about square matrices.
A matrix eigenvalue is said to be simple if its algebraic multiplicity equals one. The spectrum of a matrix is said to be simple provided that all eigenvalues of the given matrix are simple. In other words, if all eigenvalues are pairwise distinct. Certainly, the matrix with a simple spectrum is diagonalizable.
We recall that a mapping f : X ^ Y between two topological spaces is said to be open if for any open set O in X the image f (O) is open in Y. For example, if a mapping f : GLn(C) ^ GLn(C) is a surjective continuous homomorphism, then it is open [15, Theorem 5.29].
Using the topological group structure of the general linear group GLn(C), one can prove the following statement [9, Proposition 4].
Proposition 4. Let f1,f2,...,fk : GLn(C) —► GLn(C) be a finite family of self-mappings of the general linear group. Let us assume that at least one of these mappings is open with respect to the Euclidean topology. Let Ai, A2,. .., Ak be arbitrary matrices in Mn(C) and let || ■ || be a norm on Mn(C). Then, for every e > 0 there exists a finite family Aie, A2£, .. ., Ake consisting of invertible matrices with simple spectra such that the inequalities
||Ai - Aie|| < e, ||A2 - A2£|| < e,..., HAk - AkeH <e hold and the product matrix
fi(AlE)f2(A2E) ■■■ fk (Ake)
has a simple spectrum.
For the case of two mappings, we put fi and /2 to be the identity mapping and the inverse mapping, respectively:
/1 : GLn(C) GLn(C) : X X; /2 : GLn(C) GLn(C) : X X-i.
Obviously, both of these mappings are open with respect to the Euclidean topology in GLn(C). Therefore, as a consequence of the preceding proposition, we have
Corollary 1. Let A and B be matrices Mn(C) and let || • || be a norm on Mn(C). Then, for every e > 0 there exists a pair of matrices Ae and Be in GLn(C) with simple spectra such that the inequalities
IIA - АеЦ <e and IB - ВеЦ <e
hold and the product matrix AeB-i has a simple spectrum.
It is worth noting that one can use Corollary 1 for estimating the tensor rank of inverse matrices in the case when the given matrices are the factors of the Kronecker products (see [8]).
We make use of the above-mentioned results to prove the following assertion.
Proposition 5. Let A be a tensor in Cnxnx2 and let || • || be a norm on Cnxnx2 . Then, the equality
inf {ЦА - BH : B e Cnxnx2 and rank (B) = n} =0
holds, i.e., the tensor A may be approximated by tensors with tensor ranks equal to n.
Proof. We set A = [AiJA2], where Ai and A2 are square matrices of size n x n. Let us fix e > 0. Using Corollary 1, we take two invertible matrices Bi and B2 of size n x n such that the inequalities
ee ||Ai - Bi||i < - and ||A2 - B2||i < -
hold and the product matrix B2B-1 has a simple spectrum.
Let us consider the tensor B = [Bi|B2]. By Bi's criterion, since the matrix B2B-1 is diagonalizable, the tensor rank of B is equal to n. Moreover, we have the following estimation:
ЦА - B||i = ||Ai - Bi||i + A - B21|i < e.
In view of the equivalence of the norms | • | and | • | 1 , the rest is clear.
□
It is known (see [16], [17, Theorem 4.3]) that the maximum value of the tensor rank on the space Cnxnx2 is given by
mrank (n,n, 2) := max {rank (A) | A e Cnxnx2j = n +
n
,2J
where the symbol |_-J means the integer part of a real number. Therefore, mrank (2n, 2n, 2) = 3n for every n e N. This fact together with the Proposition 5 guarantees that, generally speaking, the tensor rank can leap an arbitrary large gap (see also [4, Section 4.5]). More precisely, we have
Corollary 2. Let n e N. There exists a tensor A e C2nx2nx2 with rank (A) = 3n and a sequence of tensors {Ak} C C2nx2nx2, k e N, such that rank (Ak) = 2n for every k e N and
lim Ak = A,
k—^
where the limit is taken in the Euclidean topology.
Finally, we can conclude that the tensor rank is not semicontinuous on the tensor space Cnxnx2 endowed with the Euclidean topology.
Corollary 3. Let n ^ 2. In the tensor space Cnxnx2 endowed with the Euclidean topology the set of tensors
{ T e Cnxnx2 | rank (T) < n }.
Acknowledgments. The research was funded by the subsidy allocated to Kazan Federal University for the state assignment in the sphere of scientific activities, project no. 1.13556.2019/13.1.
The authors are grateful to Professors M.M. Karchevskii, R.R. Shagidullin and the participants of the seminar "Tensor Analysis" held at Kazan Federal University for their helpful discussions of the problems covered in this paper. We thank Professors G.G. Amosov, M. Fragoulopoulou, and A.Ya. Helemskii for their discussions during the Conference "Probability Theory and Mathematical Statistics" (November 7-10, 2017, Kazan).
Author Contributions. Introduction and Section 2 were written by R.N. Gume-rov. Section 1 was written by A.S. Sharafutdinov. The results of Section 2 were proved by R.N. Gumerov.
References
1. Helemskii A.Ya. Quantum functional analysis: Non-coordinate approach. In: University Lecture Series. Providence, Rhode Island, Amer. Math. Soc., 2010, vol. 56. 241 p.
2. Hackbusch W. Tensor Spaces and Numerical Tensor Calculus. Berlin, Springer, 2012. xxiv, 500 p. doi: 10.1007/978-3-642-28027-6.
3. Landsberg J.M. Tensors: Geometry and applications. In: Graduate Studies in Mathematics. Providence, Rhode Island, Amer. Math. Soc, 2012, vol. 128. 439 p.
4. de Silva V., Lim L.-H. Tensor rank and the ill-posedness of the best low-rank approximation problem. SIAM J. Matrix Anal. Appl., 2008, vol. 30, no. 3, pp. 1084-1127. doi: 10.1137/06066518X.
5. Bini D., Capovani M., Lotti G., Romani F. O(n2'77gg) complexity for n x n approximate matrix multiplication. Inf. Process. Lett., 1979, vol. 8, pp. 234-235.
6. Gumerov R.N. Homological dimension of radical algebras of Beurling type with a rapidly decreasing weight. Moscow Univ. Math. Bull., 1988, vol. 43, no. 5, pp. 24-28.
7. Gumerov R.N. On a geometric test for nontriviality of the simplicial and cyclic cohomology of Banach algebras. Moscow Univ. Math. Bull., 1991, vol. 46, no. 3, pp. 52-53.
8. Tyrtyshnikov E.E. Tensor ranks for the inversion of tensor-product binomials. J. Comput. Appl. Math., 2010, vol. 234, no. 11, pp. 3170-3174. doi: 10.1016/j.cam.2010.02.006.
9. Gumerov R.N., Vidunov S.I. Approximation by matrices with simple spectra. Lobachevskii J. Math., 2016, vol. 37, no. 3, pp. 240-243. doi: 10.1134/S1995080216030112.
10. Greub W.H. Multilinear Algebra. Berlin, Heidelberg, Springer-Verlag, 1967. 225 p. doi: 10.1007/978-3-662-00795-2.
11. Helemskii A.Ya. Lectures and exercises on functional analysis. In: Translations of Mathematical Monographs. Providence, Amer. Math. Soc., 2006, vol. 233. 470 p.
12. Bredon G.E. Introduction to Compact Transformation Groups. New York, Acad. Press, 1972. 461 p.
13. Vries de J. Elements of Topological Dynamics. Dordrecht, Springer, 1993. xvi, 748 p. doi: 10.1007/978-94-015-8171-4.
14. Sakata T., Sumi T., Miyazaki M. Algebraic and Computational Aspects of Real Tensor Ranks. Tokyo, Springer, 2016. viii, 108 p. doi: 10.1007/978-4-431-55459-2.
15. Hewitt E., Ross K.A. Abstract Harmonic Analysis. Vol. 1. Berlin, Springer, 1963. 519 p. doi: 10.1007/978-1-4419-8638-2.
16. Ja'Ja' J. Optimal evaluation of a pair of bilinear forms. SIAM J. Comput., 1979, vol. 8, no. 3, pp. 281-293. doi: 10.1137/0208037.
17. Sumi T., Miyazaki M., Sakata T. Rank of 3-tensors with 2 slices and Kronecker canonical forms. Linear Algebra Appl., 2009, vol. 431, no. 10, pp. 1858-1868. doi: 10.1016/j.laa.2009.06.023.
Received October 25, 2017
Gumerov Renat Nelsonovich, Candidate of Physical and Mathematical Sciences, Associate Professor of the Department of Mathematical Analysis Kazan Federal University
ul. Kremlevskaya, 18, Kazan, 420008 Russia E-mail: [email protected]
Sharafutdinov Azat Shamilevich, Student of Lobachevsky Institute of Mathematics and Mechanics
Kazan Federal University
ul. Kremlevskaya, 18, Kazan, 420008 Russia E-mail: [email protected]
УДК 512.64+517.98
Аппроксимация тензорами малого ранга и топологическая группа обратимых матриц
Р.Н. Гумеров, Л.Ш. Шарафутдинов
Казанский (Приволжский) федеральный университет, г. Казань, 420008, Россия
Аннотация
Под тензором понимается элемент тензорного произведения векторных пространств над некоторым полем. С точностью до выбора базисов в множителях тензорных произведений каждый тензор может быть снабжен координатами, то есть представлен в виде массива, состоящего из чисел. Статья посвящена свойствам тензорного ранга, который является естественным обобщением понятия матричного ранга. Существенную роль в изучении играет топологическая группа обратимых матриц. Полилинейное матричное умножение обсуждается с точки зрения групп преобразований. Рассматривается вопрос об аппроксимации тензорами малого ранга в конечномерных тензорных произведениях. Показывается, что задача о наилучшем приближении тензорами ранга п не имеет решения в пространстве тензоров размера пхпх2. С этой целью используется аппроксимацию матрицами с простыми спектрами.
Ключевые слова: аппроксимация матрицами с простыми спектрами, аппроксимация тензорами малого ранга, действие группы, норма на тензорном пространстве, открытое отображение, простой спектр матрицы, топологическая группа обратимых матриц, топологическая группа преобразований, тензорный ранг
Поступила в редакцию 25.10.17
Гумеров Ренат Нельсонович, кандидат физико-математических наук, доцент кафедры математического анализа
Казанский (Приволжский) федеральный университет
ул. Кремлевская, д. 18, г. Казань, 420008, Россия E-mail: [email protected]
Ш^арафутдинов Азат Шамильевич, студент Института математики и механики им. Н.И. Лобачевского
Казанский (Приволжский) федеральный университет
ул. Кремлевская, д. 18, г. Казань, 420008, Россия E-mail: [email protected]
I For citation: Gumerov R.N., Sharafutdinov A.S. A low-rank approximation of tensors ( and the topological group structure of invertible matrices. Uchenye Zapiski Kazanskogo \ Universiteta. Seriya Fiziko-Matematicheskie Nauki, 2018, vol. 160, no. 4, pp. 788-796.
/ Для цитирования : Gumerov R.N., Sharafutdinov A.S. A low-rank approximation of ( tensors and the topological group structure of invertible matrices // Учен. зап. Казан. \ ун-та. Сер. Физ.-матем. науки. - 2018. - Т. 160, кн. 4. - С. 788-796.
