This post uses (an early version of) the Orbit system developed by Andy Matuschak.

  • Definition

    • A function is just a way of assigning exactly one output to each of its possible inputs.

  • Notation

    • If $f$ is a function and $x$ is a possible input, then the output of $f$ when the input is $x$ is written $f(x)$.
    • Often this is read as ‘f of x’.
    • A common mistake is to read this as ‘f multipled with x’! Brackets do sometimes mean multiplication, but not in this context.

  • Examples

    • $f$ is the function that adds 7 to its input. So for example, $f(3) = 10$.
    • $g$ is the function that outputs 1 if the sum of the first two inputs is greater than third input, and 0 otherwise. So for example, $g(5, 7, 10) = 1$ and $g(5, 7, 20) = 0$.
    • $b$ is the function that has inputs somebody’s weight and height, and outputs their body mass index.

An important group of functions

I won’t explain why these are important, but they occur in all numerical fields, e.g. economics or engineering or machine learning. In particular, they occur in neural networks. I will illustrate with an example.

Example

  • $f$ has three inputs, $x_1, x_2, x_3$, multiplies them by $3, 1, 4$ respectively, and adds them together.
  • So $f(1, 2, 3) = (3 \times 1) + (1 \times 2) + (4 \times 3) = 17$.

In the context of machine learning, the numbers $3, 1, 4$ are called the weights or parameters of the function.

Also in the context of neural networks, this calculation is often represented in the following diagram:

A diagram of the function $f$ with inputs $1,2,3$

For a generic input, the function would be drawn as:

A diagram of the function $f$ with inputs $x_1, x_2, x_3$

As you can see, this diagram is already a bit messy. Imagine if we had more inputs, or, we added more layers on the end! Hence, the convention is to just hide the output with the understanding we multiply each input with the corresponding weight, and then add them all together.

A diagram of the function $f$ with inputs $x_1, x_2, x_3$, with cleaned output

In this context, the circles and the numbers inside them are called neurons or nodes.

Disclaimer: There are actually two little extra bits going on in a neural network in these diagrams. First is ‘bias’ and second is an ‘activation function’. I will not explain them here as they are explained in 3Blue1Brown’s video on neural networks. The aim of this post is to explain what a function is and explain one particular function.