In the cases of real orthogonal or complex Hermitian inner product spaces with a positive definite inner product there are no null vectors so we can start from any existing basis, \(\{f_i\}\) say, and systematically construct an orthogonal basis, \(\{e_i\}\), as follows. We begin by setting \(e_1=f_1\). We set \(e_2\) to be \(f_2\) with any component parallel to \(f_1\) removed, that is,
\begin{equation*}
e_2=f_2-\pi_{e_1}f_2,
\end{equation*}
where we have introduced the operator
\begin{equation*}
\pi_uv=\frac{(u,v)}{(u,u)}u.
\end{equation*}
Likewise \(e_3\) is just \(f_3\) with its components in the \(e_1\) and \(e_2\) directions removed and so on with the general vector \(e_j\) given by
\begin{equation}
e_j=f_j-\sum_{i=1}^{j-1}\pi_{e_i}f_j.
\end{equation}
Given the orthogonal basis \(\{e_i\}\) we can then normalise each vector to obtain an orthonormal basis. This procedure, for constructing an orthogonal basis from any given basis in an inner product space, is known as Gram-Schmidt orthogonalisation.

We can also view the construction ‘in reverse’ as follows. Given the assumption of positive (negative) definite inner product we know that not only is the \(n\times n\) matrix, \(\mathbf{G}\), of the inner product with respect to the given basis, \(\{f_i\}\), invertible, but every \(k\times k\) submatrix with elements \(G_{ij}\), \(1\leq i,j\leq k\) is also invertible. Indeed, if it weren’t, and there were numbers \(x^i\) not all zero such that \(\sum_{i=1}^kG_{ij}x^j=0\), then the vector \(\sum_{i=1}^kx^if_i\) would a non-zero null vector since \((\sum_{i=1}^kx^if_i,\sum_{j=1}^kx^jf_j)=\sum_{i,j=1}^k{x^i}^*x^j(f_i,f_j)=\sum_{i,j=1}^k{x^i}^*x^jG_ij=0\). Now define \(e_n=f_n-\sum_{i,j=1}^{n-1}G_{ij}^{-1}(f_j,f_n)f_i\). It is clearly orthogoonal to all \(f_i\), \(1\leq i\leq n-1\). \(e_{n-1}\) is then defined as \(e_{n-1}=f_{n-1}-\sum_{i,j=1}^{n-2}G_{ij}^{-1}(f_j,f_{n-1})f_i\), and is clearly orthogonal to all \(f_i\), \(1\leq i\leq n-2\) and to \(e_n\). Continuing in this way we arrive at the desired orthogonal basis.

Now, we know that if \(U\) is a subspace of \(V\) on which the restriction of the inner product is non-degenerate then we can write, \(V=U\oplus U^\perp\), which specifies the orthogonal projection onto \(U\), \(P:V\mapto U\). In fact, \(Pv\) is the vector in \(U\) closest to \(v\) in the sense that \(\norm{v-Pv}\leq\norm{v-u}\) for all \(u\in U\) with equality if and only if \(u=Pv\). To see this, observe first that for any \(v\in V\) and \(u\in U\), we have \(v-Pv\in U^\perp\) and \(Pv-u\in U\), so that \((v-Pv,Pv-u)=0\), and therefore,
\begin{align*}
\norm{v-u}^2&=\norm{v-Pv+Pv-u}^2\\
&=(v-Pv+Pv-u,v-Pv+Pv-u)\\
&=(v-Pv,v-Pv)+(Pv-u,Pv-u)+2\Real(v-Pv,Pv-u)\\
&=\norm{v-Pv}^2+\norm{Pv-u}^2,
\end{align*}
from which it follows that \(\norm{v-Pv}\leq\norm{v-u}\) with equality if and only if \(u=Pv\).

In the context of the Gram-Schmidt procedure, if \(U=\Span(e_1,\dots,e_k)\), then notice that the orthogonal projector \(P\) is just \(P=\sum_{i=1}^k\pi_{e_i}\), so geometrically, the inductive step of the Gram-Schmidt procedure is to express the next of the original basis vectors, \(f_{k+1}\), as the sum of the vector in \(\Span(e_1,\dots,e_k)\) closest to \(f_{k+1}\) with an element \(e_{k+1}\) of \(\Span(e_1,\dots,e_k)^\perp\).

It’s worth mentioning that any set of orthonormal vectors in a real orthogonal or complex Hermitian inner product space with positive definite inner product, may be extended to an orthonormal basis for \(V\) since they can be extended to a basis of \(V\) and Gram-Schmidt employed to orthogonalise the extension.