What the answer is to be

$8.62
Let x represent the cost of one of the books purchased separately. The problem statement tells us the cost of 3 books will be ...
... 3x = 20.40 + 5.45
... x = 25.85/3 ≈ 8.62
The cost of an individual book purchased separately is $8.62.
_____
Comment on the problem statement
The problem statement is not self-consistent. If you purchase 3 books at $8.62 each, the total price will be $25.86, which is $5.46 more than the price of the set. (not $5.45 more)