TENSOR PRODUCT

In mathematics, the 'tensor product', denoted by $otimes$, may be applied in different contexts to vectors, matrices, tensors, vector spaces, algebras, topological vector spaces, and modules. In each case the significance of the symbol is the same: the most general bilinear operation. In some contexts, this product is also referred to as 'outer product'.
'Example:'
:$$
mathbf{b} otimes mathbf{a}
ightarrow
egin{bmatrix}b_1 \ b_2 \ b_3 \ b_4end{bmatrix}
egin{bmatrix}a_1 & a_2 & a_3end{bmatrix} =
egin{bmatrix}a_1b_1 & a_2b_1 & a_3b_1 \ a_1b_2 & a_2b_2 & a_3b_2 \ a_1b_3 & a_2b_3 & a_3b_3 \ a_1b_4 & a_2b_4 & a_3b_4end{bmatrix}
Resultant rank = 2, resultant dimension = 4×3 = 12.
Here rank denotes the tensor rank (number of requisite indices), while dimension counts the number of degrees of freedom in the resulting array; the matrix rank is 1.
A representative case is the Kronecker product of any two rectangular arrays, considered as matrices. A dyadic product is the special case of the tensor product between two vectors of the same dimension.

Contents

Tensor product of two tensors

Example

Kronecker product of two matrices

Tensor product of multilinear maps

Tensor product of vector spaces

Universal property of tensor product

Tensor product of Hilbert spaces

Definition

Properties

Examples and applications

Relation with the dual space

Types of tensors, e.g., alternating

Over more general rings

Tensor product for computer programmers

Notes

See also

Tensor product of two tensors

There is a general formula for the components of a product of two (or more) tensors. For example, if ''U'' and ''V'' are two covariant tensors of rank ''n'' and ''m'' (respectively), then the components of their tensor product are given by
:$(Votimes\; U)\_\{i\_1i\_2...i\_\{m+n\}\}\; =\; V\_\{i\_1i\_2i\_3...i\_n\}U\_\{i\_\{n+1\}i\_\{n+2\}...i\_\{n+m\}\}$.^[1]
Thus, the components of the tensor product of two tensors are the ordinary product of the components of each tensor.
Note that in the tensor product, the factor ''U'' consumes the first rank(''U'') indices, and the factor ''V'' consumes the next rank(''V'') indices, so
:$mathrm\{rank\}(\; U\; otimes\; V\; )=mathrm\{rank\}(U)+mathrm\{rank\}(V)$

Example

Let 'U' be a tensor of type (1,1) with components ''U^α_β'', and let 'V' be a tensor of type (1,0) with components ''V^γ''. Then
:
and
:$V^mu\; U^$
u {}_sigma = (V otimes U)^{mu
u} {}_sigma .
The tensor product inherits all the indices of its factors.
See also: Classical treatment of tensors

Kronecker product of two matrices

''Main article: Kronecker product.''
With matrices this operation is usually called the ''Kronecker product'', a term used to make clear that the result has a particular block structure imposed upon it, in which each element of the first matrix is replaced by the second matrix, scaled by that element. For matrices $U$ and $V$ this is:
:$U\; otimes\; V$
= egin{bmatrix} u_{11}V & u_{12}V & cdots \
u_{21}V & u_{22}V \
dots & & ddots
end{bmatrix}
= egin{bmatrix}
u_{11}v_{11} & u_{11}v_{12} & cdots & u_{12}v_{11} & u_{12}v_{12} & cdots \
u_{11}v_{21} & u_{11}v_{22} & & u_{12}v_{21} & u_{12}v_{22} \
dots & & ddots \
u_{21}v_{11} & u_{21}v_{12} \
u_{21}v_{21} & u_{21}v_{22} \
dots
end{bmatrix}.

Tensor product of multilinear maps

Given multilinear maps $f(x\_1,...x\_k)$ and $g(x\_1,...\; x\_m)$
their tensor product is the multilinear function
:$(f\; otimes\; g)\; (x\_1,...,x\_\{k+m\})=f(x\_1,...,x\_k)g(x\_\{k+1\},...,x\_\{k+m\})$

Tensor product of vector spaces

The tensor product $V\; otimes\; W$ of two vector spaces ''V'' and ''W'' over a field $K$ has a formal definition by the method of ''generators and relations''. The equivalence class under these relations (given below) of $(v,w)$ is called a ''tensor'' and is denoted by $v\; otimes\; w$. By construction, one can prove several identities between tensors and form an algebra of tensors.
To construct $V\; otimes\; W$, take a vector space over $K$ with basis $V\; imes\; W$ and apply (factor out the subspace generated by) the following multilinear relations:

★ $(v\_1+v\_2)otimes\; w=v\_1otimes\; w+v\_2otimes\; w$

★ $votimes\; (w\_1+w\_2)=votimes\; w\_1+votimes\; w\_2$

★ $cvotimes\; w=votimes\; cw=c(votimes\; w)$
where $v,v\_i,w,w\_i$ are vectors from the appropriate spaces, and $c$ is from the underlying field $K$.
We can then derive the identity
:$0votimes\; w=votimes\; 0w=0(votimes\; w)=0$,
the zero in $V\; otimes\; W$.
The resulting tensor product $V\; otimes\; W$ is itself a vector space, which can be verified by directly checking the vector space axioms.
Given bases $\{v\_i\}$ and $\{w\_i\}$ for ''V'' and ''W'' respectively, the tensors of the form $v\_i\; otimes\; w\_j$
forms a basis for $V\; otimes\; W$. The dimension of the tensor product therefore is the product of dimensions of the original spaces; for instance $mathbb\{R\}^m\; otimes\; mathbb\{R\}^n$ will have dimension $mn$.

Universal property of tensor product

The tensor product is characterized by a universal property. Consider the problem of embedding the Cartesian product ''V'' × ''W'' into a vector space ''X'' via a bilinear map ''φ''. The tensor product construction ''V'' ⊗ ''W'', together with the natural embedding map ''φ'' : ''V'' × ''W'' → ''V'' ⊗ ''W'' given by
:$phi\; (u,w)=\; u\; otimes\; w\; ,$
is the "universal" solution to this problem in the following sense. For any other such pair (''X'', ''ψ''), where ''X'' is a vector space, and ψ a bilinear mapping ''V'' × ''W'' → ''X'', there exists an ''unique'' linear map
:$T\; :\; V\; otimes\; W$
ightarrow X
such that
:$psi\; =\; T\; circ\; phi.$
Assuming this universal property, it can be readily verified that the tensor product is unique up to isomorphism.
An immediate consequence is the identification of
:$B(V\; imes\; W,\; X),$
the bilinear maps from ''V'' × ''W'' to ''X'' and the linear maps
:$L(V\; otimes\; W,\; X).$
The natural isomorphism maps ''ψ'' to ''T''.

Tensor product of Hilbert spaces

The tensor product of two Hilbert spaces is another Hilbert space, which is defined as described below.

Definition

The discussion so far has been purely algebraic. In light of the extra structure on Hilbert spaces, one would like to introduce an inner product, and therefore a topology, on the tensor product that arise naturally from those of the factors. Let ''H''₁ and ''H''₂ be two Hilbert spaces with inner products $langle\; cdot,cdot$
angle_1 and $langle\; cdot,cdot$
angle_2, respectively. Construct the tensor product of ''H''₁ and ''H''₂ as vector spaces as explained above. We can turn this vector space tensor product into an inner product space by defining
:$langlephi\_1otimesphi\_2,psi\_1otimespsi\_2$
angle = langlephi_1,psi_1
angle_1 , langlephi_2,psi_2
angle_2 quad mbox{for all } phi_1,psi_1 in H_1 mbox{ and } phi_2,psi_2 in H_2
and extending by linearity. That this inner product is the natural one is justified by the identification of scalar-valued bilinear maps on ''H''₁ × ''H''₂ and linear functionals on their vector space tensor product. Finally, take the completion under this inner product. The resulting Hilbert space is the tensor product of  ''H''₁ and ''H''₂.

Properties

If ''H''₁ and ''H''₂ have orthonormal bases {φ_''k''} and {ψ_''l''}, respectively, then {φ_''k'' ⊗ ψ_''l''} is an orthonormal basis for ''H''₁ ⊗ ''H''₂.

Examples and applications

The following examples show how tensor products arise naturally.
Given two measure spaces ''X'' and ''Y'', with measures μ and ν respectively, one may look at L²(''X'' × ''Y''), the space of functions on ''X'' × ''Y'' that are square integrable with respect to the product measure μ × ν. If ''f'' is a square integrable function on ''X'', and ''g'' is a square integrable function on ''Y'', then we can define a function ''h'' on ''X'' × ''Y'' by ''h''(''x'',''y'') = ''f''(''x'') ''g''(''y''). The definition of the product measure ensures that all functions of this form are square integrable, so this defines a bilinear mapping L²(''X'') × L²(''Y'') → L²(''X'' × ''Y''). Linear combinations of functions of the form ''f''(''x'') ''g''(''y'') are also in L²(''X'' × ''Y''). It turns out that the set of linear combinations is in fact dense in L²(''X'' × ''Y''), if L²(''X'') and L²(''Y'') are separable. This shows that L²(''X'') ⊗ L²(''Y'') is isomorphic to L²(''X'' × ''Y''), and it also explains why we need to take the completion in the construction of the Hilbert space tensor product.
Similarly, we can show that L²(''X''; ''H''), denoting the space of square integrable functions ''X'' → ''H'', is isomorphic to L²(''X'') ⊗ ''H'' if this space is separable. The isomorphism maps ''f''(''x'') ⊗ φ ∈ L²(''X'') ⊗ ''H'' to ''f''(''x'')φ ∈ L²(''X''; ''H''). We can combine this with the previous example and conclude that L²(''X'') ⊗ L²(''Y'') and L²(''X'' × ''Y'') are both isomorphic to L²(''X''; L²(''Y'')).
Tensor products of Hilbert spaces arise often in quantum mechanics. If some particle is described by the Hilbert space ''H''₁, and another particle is described by ''H''₂, then the system consisting of both particles is described by the tensor product of ''H''₁ and ''H''₂. For example, the state space of a quantum harmonic oscillator is L²('R'), so the state space of two oscillators is L²('R') ⊗ L²('R'), which is isomorphic to L²('R'²). Therefore, the two-particle system is described by wave functions of the form φ(''x''₁, ''x''₂). A more intricate example is provided by the Fock spaces, which describe a variable number of particles.

Relation with the dual space

In the discussion on the universal property, replacing ''X'' by the underlying scalar field of ''V'' and ''W'' yields that the space $(V\; otimes\; W)^star$ (the dual space of $V\; otimes\; W$, containing all linear functionals on that space) is naturally identified with the space of all
bilinear functionals on $V\; imes\; W$. In other words, every bilinear functional is a functional
on the tensor product, and vice versa.
Whenever $V$ and $W$ are finite dimensional, there is a natural isomorphism between $V^star\; otimes\; W^star$ and $(V\; otimes\; W)^star$, whereas for vector spaces of arbitrary dimension we only have an inclusion $V^star\; otimes\; W^starsubset\; (V\; otimes\; W)^star$.
So, the tensors of the linear functionals are bilinear functionals. This
gives us a new way to look at the space of bilinear functionals, as a tensor
product itself.

Types of tensors, e.g., alternating

Linear subspaces of the bilinear
operators (or in general, multilinear operators) determine natural quotient spaces of the tensor space, which are frequently useful. See wedge product for the first major example. Another would be the treatment of algebraic forms as symmetric tensors.

Over more general rings

''See tensor product of modules over a ring''

Tensor product for computer programmers

===Array programming languages===
Array programming languages may have this pattern built in. For example, in APL the tensor product is expressed as $circ\; .\; imes$ (for example $A\; circ\; .\; imes\; B$ or $A\; circ\; .\; imes\; B\; circ\; .\; imes\; C$). In J the tensor product is the dyadic form of '
★ /' (for example 'a
★ / b' or ' a
★ / b
★ / c').
Note that J's treatment also allows the representation of some tensor fields (as 'a' and 'b' may be functions instead of constants -- the result is then a derived function, and if 'a' and 'b' are differentiable, then 'a
★ /b' is differentiable).
However, these kinds of notation are not universally present in array languages. Other array languages may require explicit treatment of indices (for example, Matlab), and/or may not support higher-order functions such as the Jacobian derivative (for example, Fortran/APL).

Notes

1.
Analogous formulas also hold for contravariant tensors, as well as tensors of mixed variance. Although in many cases such as when there is an inner product defined, the distinction is irrelevant.

See also

★ Outer product

★ tensor product of modules

★ tensor product of R-algebras

★ tensor product of fields

★ tensor product of Hilbert spaces

★ topological tensor product

★ tensor product of line bundles

★ tensor product of graphs

★ tensor product of quadratic forms

★ dyadic product