Scalars – Understanding Scalars (Zero-Order Tensors)

What is a Scalar?

A scalar is the simplest form of a tensor. It’s a single number, without direction. Scalars contrast with higher order tensors like vectors (1st order), matrices (2nd order), and so on. In other words, a scalar has zero dimensions.

Why are Scalars Important?

Foundation of Math Operations: When we work with high-dimensional data, the operations often boil down to scalar computations. For instance, when you multiply two matrices, the individual operations involve multiplying scalars.

Understanding Basic Properties: Concepts like magnitude, units, and identity elements are best understood using scalars before they’re applied to vectors and matrices.

Performance Metrics: In machine learning, metrics such as loss, accuracy, or precision are often represented as scalars.

Interactions with Higher-Order Tensors

Scalars frequently interact with vectors, matrices, and higher-order tensors. For instance:

Scalar Multiplication: Multiplying a matrix by a scalar involves multiplying each element of the matrix by the scalar.

NumPy:

matrix = np.array([[1, 2], [3, 4]])
result = matrix * 5
print(result)  

[[ 5 10]
 [15 20]]

Advanced Properties of Scalars

Identity: The number 1 is often called a multiplicative identity because multiplying any number by 1 doesn’t change that number. Likewise, 0 is an additive identity because adding 0 to a number doesn’t change it.

Inverse: Every scalar has a multiplicative inverse, such that when it’s multiplied by its inverse, the result is 1. For example, the inverse of 5 is 1/5

Absolute Value: It represents the magnitude of a scalar. In many libraries, it’s computed using the abs function.

Scalars in Different Frameworks

We will be illustrating scalar operations in three prominent libraries:

• NumPy
• PyTorch
• TensorFlow

Scalars in NumPy

import numpy as np

# Creating a scalar
scalar = np.array(5)
print(scalar)  # Output: 5

# Checking its dimensions
print(scalar.shape)  # Output: ()

5
()

Scalars in PyTorch

import torch

# Creating a scalar
scalar = torch.tensor(5)
print(scalar)  # Output: tensor(5)

# Checking its dimensions
print(scalar.size())  # Output: torch.Size([])

tensor(5)
torch.Size([])

Scalars in TensorFlow

import tensorflow as tf

# Creating a scalar
scalar = tf.constant(5)
print(scalar)  # Output: tf.Tensor(5, shape=(), dtype=int32)

# Checking its dimensions
print(scalar.shape)  # Output: ()

tf.Tensor(5, shape=(), dtype=int32)
()

Scalar Operations

Addition

NumPy:

result = np.add(5, 3)
print(result)  # Output: 8

8

PyTorch:

result = torch.add(torch.tensor(5), torch.tensor(3))
print(result)  # Output: tensor(8)

tensor(8)

TensorFlow:

result = tf.add(tf.constant(5), tf.constant(3))
print(result)  # Output: tf.Tensor(8, shape=(), dtype=int32)

tf.Tensor(8, shape=(), dtype=int32)

Multiplication

NumPy:

result = np.multiply(5, 3)
print(result)  # Output: 15

15

PyTorch:

result = torch.mul(torch.tensor(5), torch.tensor(3))
print(result)  # Output: tensor(15)

tensor(15)

TensorFlow:

result = tf.multiply(tf.constant(5), tf.constant(3))
print(result)  # Output: tf.Tensor(15, shape=(), dtype=int32)

tf.Tensor(15, shape=(), dtype=int32)

Division

NumPy:

result = np.divide(10, 2)

print(result)  # Output: 5.0

5.0

PyTorch:

result = torch.div(torch.tensor(10), torch.tensor(2))
print(result)  # Output: tensor(5)

tensor(5.)

TensorFlow:

result = tf.divide(tf.constant(10), tf.constant(2))
print(result)  # Output: tf.Tensor(5.0, shape=(), dtype=float64)

tf.Tensor(5.0, shape=(), dtype=float64)

Conclusion

More you understand about scalars, the more solid your foundational knowledge will be as you delve into vectors, matrices, and more advanced multi-dimensional tensors. Whether you’re working with NumPy, TensorFlow, or PyTorch, remember the essential role these zero-order tensors play in complex operations and computations.