Let $f(x)=\sqrt{x+1}$ and $g(x)=x^{2}$. We can define a new function $h$ as
$h(x)=f(g(x))=f(x^{2})=\sqrt{x^{2}+1}.$ That is, to obtain $h(x)$, we substitute $g(x)$ for $x$ in the expression $f(x)$.

To generalize this process suppose $f$ and $g$ are any two given functions. We start with a number $x$ in the domain of $g$ and apply $g$ to it to get $g(x)$, then we apply $f$ to $g(x)$ and thereby obtain the number $f(g(x))$. Obviously this process presupposes that it makes sense to calculate $f$ at the point $g(x)$. In other words, the new function is defined only if $g(x)$ is in the domain of $f$ (otherwise we cannot use $g(x)$ as the input for $f$).

Consecutive application of functions is known as composition of functions. The new function that takes $x$ and assigns to it the value $f(g(x))$ is often denoted by $f\circ g$. The symbol $f\circ g$ is read “$f$ circle $g$.” Figure 1(a) shows the composition $f\circ g$ as a machine diagram and Figure 1(b) illustrates it as an arrow diagram.

(a) (b) Fig 1(a) Machine digram for $f\circ g$. A composite (or composition) function $f\circ g$ applies the $g$ machine to the input $x$ and then uses the output $g(x)$ as the input for the $f$ machine.
Fig. 1(b)  Arrow diagram for $f\circ g$. If $x$ is in the domain of $g$ and $g(x)$ is in the domain of $f$, then we may compose $f$ and $g$ to form $f\circ g$.
Let $f$ and $g$ be two functions. For those points $x$ in the domain of $g$ for which $g(x)$ lies in the domain of $f$, the composite function $f\circ g$ is defined by
$(f\circ g)(x)=f(g(x)).$
• In set notation, the domain of $f\circ g$ is
$\bbox[#F2F2F2,5px,border:2px solid black]{Dom(f\circ g)=\{x|\ x\in Dom(g)\text{ and }g(x)\in Dom(f)\}.}$
• Note that the order of composition of two functions matters. In calculating $(f\circ g)(x)$, we first evaluate $g$ at $x$ and then use $g(x)$ as the input to calculate $f$ of the result. But in calculating $(g\circ f)(x)$, we first evaluate $f$ at $x$ and then calculate $g$ at the the point $f(x)$. Therefore, $(f\circ g)(x)$ is often quite different from $(g\circ f)(x)$.
Example
Let $f(x)=\sqrt{x}$ and $g(x)=1+x^{2}$. Find the following functions and their domains.
(a) $f\circ g$
(b) $g\circ f$
Solution
We have
$Dom(f)=\{x|\ x\geq0\}=[0,\infty),\qquad Dom(g)=\mathbb{R}.$ (a)
\begin{align*}
(f\circ g)(x) & =f(g(x))\\
& =f(1+x^{2})\\
& =\sqrt{1+x^{2}}.
\end{align*}
Because $g(x)=1+x^{2}\geq1$ for all $x$, $g(x)$ is always in the domain of $f$. Therefore
$Dom(f\circ g)=\mathbb{R}.$ (b)
\begin{align*}
(g\circ f)(x) & =g(f(x))\\
& =g(\sqrt{x})\\
& =1+(\sqrt{x})^{2}\\
& =1+x  &{\footnotesize (\text{ if } x\geq 0)}
\end{align*}
Because the domain of $g$ is the entire set of real numbers, $f(x)$ always lies in the domain of $g$. However, $f$ is defined only for $x\geq0$. Therefore, the domain of $g\circ f$ is the same as the domain of $f$, namely
$Dom(g\circ f)=Dom(f)=\{x|\ x\geq0\}=[0,\infty).$
Example
If $f(x)=\dfrac{x+1}{x-1}$, find the function $f\circ f$ and its domain.
Solution
Let’s first find $f\circ f$
\begin{align*}
(f\circ f)(x) & =f(f(x))\\
& =f\left(\dfrac{x+1}{x-1}\right)\\
& =\dfrac{\dfrac{x+1}{x-1}+1}{\dfrac{x+1}{x-1}-1}\\
& =\dfrac{\dfrac{x+1+x-1}{x-1}}{\dfrac{x+1-(x-1)}{x-1}}\\
& =\frac{2x}{2}=x
\end{align*}
Just by looking at the above formula one may be tempted to say that the domain of $f\circ f$ is $\mathbb{R}$. However, the domain of $f\circ f$ consists of all $x$ that
(1) $x$ lies in the domain of $f$,
(2) $f(x)$ lies in the domain of $f$.

The domain of $f$ is all real numbers except $x=1$ (recall that division by zero is not defined). So to satisfy the first condition we have to exclude $x=1$. To satisfy the second condition, we also have to exclude $x$ for which
$\frac{x+1}{x-1}=1.$ It turns out that this equation has no solution because it is equivalent to $x+1=x-1$ or $-1=1$. This means that $f(x)$ is always in the domain of $f$, and the second conditions is automatically satisfied without any further restriction on $x$. Thus in this specific example
$Dom(f\circ f)=Dom(f)=\{x|\ x\neq1\}=\mathbb{R}-\{1\}.$

Example
Find functions $f$ and $g$ such that $h=f\circ g$ given that
$h(x)=\cos(x^{2}).$
Solution
To calculate $h(x)$ we first square $x$ and then take the cosine of $x^{2}$. We can therefore set
$g(x)=x^{2} \qquad &{\footnotesize(\text{square the input})}$ and
$f(x)=\cos x \qquad &{\footnotesize (\text{ take cosine of the input})}$ We observe that
$(f\circ g)(x)=f(g(x))=f(x^{2})=\cos(x^{2}).$
We can combine three or more functions by composing them two at a time. For example, the function $F$ given by
$F(x)=\frac{1}{2+x^{2}},$ is a composition of three functions $F=f\circ g\circ h$, where
$h(x)=x^{2},\qquad g(x)=x+2,\qquad f(x)=\frac{1}{x}.$ Notice that we can compose $f$ and $g$ first then compose $f\circ g$ with $h$ or we can first compose $g$ and $h$ and then compose $f$ and $g\circ h$. In general, composition is associative. That is
$f\circ g\circ h=(f\circ g)\circ h=f\circ(g\circ h)$
Example
Find three functions $f,g,$ and $h$ such that $F=f\circ g\circ h$ given that
$F(x)=\sqrt{|x|+3}.$
Solution
To calculate $F(x)$, we fist take the absolute value of $x$, add 3, and then take the square root. We can therefore set
$h(x)=|x|,$ $g(x)=x+3,$ and
$f(x)=\sqrt{x}.$ With this choice of $f,g,$ and $h$, we observe that
$(f\circ g\circ h)(x)=f\left(g\left(h(x)\right)\right)=f\left(g\left(|x|\right)\right)=f\left(|x|+3\right)=\sqrt{|x|+3}.$