Let the given vectors be ( mathbf{a} ) and ( mathbf{b} )
The unit vector ( mathbf{u} ) that maximizes ( lVert mathbf{u} times mathbf{a} rVert ) must be perpendicular to ( mathbf{a} )
If maximizing ( lVert mathbf{u} times mathbf{a} rVert ) subject to ( lVert mathbf{u} rVert = 1 ), then any unit vector orthogonal to ( mathbf{a} ) is a maximizer
If maximizing ( lVert mathbf{u} times mathbf{a} rVert + lVert mathbf{u} times mathbf{b} rVert ), the maximizing unit vector is along the direction of ( mathbf{a} times mathbf{b} )
The unit vector is
( mathbf{u} = pm dfrac{mathbf{a} times mathbf{b}}{lVert mathbf{a} times mathbf{b} rVert} )
If ( mathbf{a} times mathbf{b} = mathbf{0} ), any unit vector perpendicular to both is not available; choose any unit vector orthogonal to the common direction
To find it directly
Compute ( mathbf{a} times mathbf{b} )
Normalize it by dividing by its magnitude
Use either sign, since both maximize the cross product magnitude