Vectors – Vector Transposition, Norms, and Unit Vectors: A Comprehensive Guide

In the world of data science, vectors play a vital role. They are fundamental to machine learning, data analysis, and artificial intelligence.

This post aims to break down the concepts of vectors, vector transposition, norms, and unit vectors, explaining their importance in the data science landscape.

Vectors

Vectors, also referred to as first-order tensors, are mathematical objects used to represent quantities that have both magnitude and direction. They are an ordered list of numbers that can describe anything from motion to the weights in a neural network.

$ F = \begin{bmatrix}
v1 \\
v2 \\
v3 \\
\end{bmatrix} $

or

$ F = \begin{bmatrix}
v1 & v2 & v3 \
\end{bmatrix} $

Properties of Vectors

• Commutativity: A + B = B + A

• Associativity: (A + B) + C = A + (B + C)

• Distributive Property: a(A + B) = aA + aB, where ‘a’ is a scalar.

1. Basic Operations with Vectors

• Addition: When you add vectors, you place them head to tail and draw the resultant vector from the tail of the first vector to the head of the last.

• Subtraction: This is equivalent to adding a negative vector.

• Scalar Multiplication: Multiplying a vector by a scalar changes its magnitude but not its direction.

• Dot Product (Scalar Product): The result is a scalar. For two vectors A and B, A·B = |A||B|cos(θ), where θ is the angle between the vectors.

• Cross Product (Vector Product): The result is a vector. For vectors A and B in 3D, the magnitude is |A||B|sin(θ), and the direction is perpendicular to both A and B, following the right-hand rule.

Addition of Vectors

import numpy as np
import torch
import tensorflow as tf

# NumPy
v1 = np.array([1,2,3])
v2 = np.array([4,5,6])
result = v1 + v2
print("NumPy result : {}".format(result))

# PyTorch
v1 = torch.tensor([1,2,3])
v2 = torch.tensor([4,5,6])
result = v1 + v2
print("PyTorch result : {}".format(result))

# TensorFlow
v1 = tf.constant([1,2,3])
v2 = tf.constant([4,5,6])
result = v1 + v2
print("TensorFlow result : {}".format(result))

NumPy result : [5 7 9]
PyTorch result : tensor([5, 7, 9])
TensorFlow result : [5 7 9]

Subtraction of Vectors

# NumPy
v1 = np.array([1,2,3])
v2 = np.array([4,5,6])
result = v1 - v2
print("NumPy result : {}".format(result))

# PyTorch
v1 = torch.tensor([1,2,3])
v2 = torch.tensor([4,5,6])
result = v1 - v2
print("PyTorch result : {}".format(result))

# TensorFlow
v1 = tf.constant([1,2,3])
v2 = tf.constant([4,5,6])
result = v1 - v2
print("TensorFlow result : {}".format(result))

NumPy result : [-3 -3 -3]
PyTorch result : tensor([-3, -3, -3])
TensorFlow result : [-3 -3 -3]

Scalar Multiplication

# NumPy
v = np.array([1,2,3])
result = 2 * v
print("NumPy result : {}".format(result))

# PyTorch
v = torch.tensor([1,2,3])
result = 2 * v
print("PyTorch result : {}".format(result))

# TensorFlow
v = tf.constant([1,2,3])
result = 2 * v
print("TensorFlow result : {}".format(result))

NumPy result : [2 4 6]
PyTorch result : tensor([2, 4, 6])
TensorFlow result : [2 4 6]

Dot Product (Scalar Product)

# NumPy
v1 = np.array([1,2,3])
v2 = np.array([4,5,6])
result = np.dot(v1, v2)
print("NumPy result : {}".format(result))

# PyTorch
v1 = torch.tensor([1,2,3])
v2 = torch.tensor([4,5,6])
result = torch.dot(v1, v2)
print("PyTorch result : {}".format(result))

# TensorFlow
v1 = tf.constant([1,2,3])
v2 = tf.constant([4,5,6])
result = tf.tensordot(v1, v2, axes=1)
print("TensorFlow result : {}".format(result))

NumPy result : 32
PyTorch result : 32
TensorFlow result : 32

Cross Product (Vector Product)

# NumPy
v1 = np.array([1,2,3])
v2 = np.array([4,5,6])
result = np.cross(v1, v2)
print("NumPy result : {}".format(result))

# PyTorch
v1 = torch.tensor([1,2,3])
v2 = torch.tensor([4,5,6])
result = torch.cross(v1, v2)
print("PyTorch result : {}".format(result))

NumPy result : [-3  6 -3]
PyTorch result : tensor([-3,  6, -3])

2. Vector Transposition

Transposing a vector means flipping its orientation. If you have a row vector, transposing it will turn it into a column vector, and vice versa.

The transposition of vector V is denoted by $V^{T}$

Transposition is commonly used in matrix multiplication and finding correlations between variables.

$ V = \begin{bmatrix}
v1 \\
v2 \\
v3 \\
\end{bmatrix} $

$ V^{T}= \begin{bmatrix}
v1 &
v2 &
v3 \
\end{bmatrix}$

Transposition is commonly used in matrix multiplication and finding correlations between variables.

# NumPy
vector = np.array([[1, 2, 3]])
transposed_vector = vector.T
print("Row Vector : {}".format(vector))
print("NumPy result : {}".format(transposed_vector))

# PyTorch
vector = torch.Tensor([[1, 2, 3]])
transposed_vector = vector.T
print("PyTorch result : {}".format(transposed_vector))

# TensorFlow
vector = tf.constant([[1, 2, 3]])
transposed_vector = tf.transpose(vector)
print("TensorFlow result : {}".format(transposed_vector))

Row Vector : [[1 2 3]]
NumPy result : [[1]
 [2]
 [3]]
PyTorch result : tensor([[1.],
        [2.],
        [3.]])
TensorFlow result : [[1]
 [2]
 [3]]

3. Norms

The norm of a vector provides a measure of its length or magnitude. In data science, norms are useful for understanding the scale of vectors, like the weights in a neural network or the error in a regression model.

L1 and L2 Norms

The two most common norms are the L1 and L2 norms:

L1 Norm: Sum of absolute values of the components.

$∣∣V∣∣1 = ∣v1| + ∣v2∣ + ∣v3∣$

L2 Norm: Square root of the sum of squared components (Euclidean norm).

$∣∣V∣∣2 = \sqrt[]{v_1^2 + v_2^2 +v_3^2}$

# Numpy

vec = np.array([1, -2, 3])

# L1 norm
l1_norm_numpy = np.linalg.norm(vec, ord=1)
print(f"L1 norm using numpy: {l1_norm_numpy}")

# L2 norm
l2_norm_numpy = np.linalg.norm(vec, ord=2)
print(f"L2 norm using numpy: {l2_norm_numpy}")

L1 norm using numpy: 6.0
L2 norm using numpy: 3.7416573867739413

# PyTorch

vec_torch = torch.tensor([1, -2, 3], dtype=torch.float32)

# L1 norm
l1_norm_pytorch = torch.norm(vec_torch, p=1)
print(f"L1 norm using PyTorch: {l1_norm_pytorch.item()}")

# L2 norm
l2_norm_pytorch = torch.norm(vec_torch, p=2)
print(f"L2 norm using PyTorch: {l2_norm_pytorch.item()}")

L1 norm using PyTorch: 6.0
L2 norm using PyTorch: 3.7416574954986572

# TensorFlow

vec_tf = tf.constant([1, -2, 3], dtype=tf.float32)

# L1 norm
l1_norm_tensorflow = tf.norm(vec_tf, ord=1)
print(f"L1 norm using TensorFlow: {l1_norm_tensorflow.numpy()}")

# L2 norm
l2_norm_tensorflow = tf.norm(vec_tf, ord=2)
print(f"L2 norm using TensorFlow: {l2_norm_tensorflow.numpy()}")

L1 norm using TensorFlow: 6.0
L2 norm using TensorFlow: 3.7416574954986572

4. Unit Vectors

A unit vector is a vector with a length or norm of 1. Unit vectors are important for providing direction without a specific magnitude. They are used in various data science applications, such as feature scaling and direction cosines.

To convert a vector into a unit vector, divide it by its magnitude:

$\hat{V} = \frac{V}{||V||}$

# Numpy

# Create a vector
vector = np.array([1, 2, 3])

# Normalize the vector to get the unit vector
unit_vector = vector / np.linalg.norm(vector)

print("NumPy Unit Vector:", unit_vector)

NumPy Unit Vector: [0.26726124 0.53452248 0.80178373]

# PyTorch

# Create a vector
vector = torch.tensor([1.0, 2.0, 3.0])

# Normalize the vector to get the unit vector
unit_vector = vector / torch.norm(vector)

print("PyTorch Unit Vector:", unit_vector)

PyTorch Unit Vector: tensor([0.2673, 0.5345, 0.8018])

# TensorFlow

# Create a vector
vector = tf.constant([1.0, 2.0, 3.0])

# Normalize the vector to get the unit vector
unit_vector = vector / tf.norm(vector)

# For TensorFlow 2.x:
print("TensorFlow Unit Vector:", unit_vector.numpy())

TensorFlow Unit Vector: [0.26726124 0.5345225  0.8017837 ]

Conclusion

Vectors, or first-order tensors, are foundational to understanding higher-dimensional tensors and are omnipresent in various scientific computations. Their operations, such as transposition, finding norms, and deriving unit vectors, are fundamental to advanced mathematics and physics.

Understanding these concepts is pivotal for anyone diving into linear algebra, data science, quantum mechanics, and numerous other fields.