The first book of the Children of Time series by Adrian Tchaikovsky (which is all that I have read) heavily features mathematics. In it, a brilliant but arrogant scientist's experiment to rapidly evolve monkeys into an advanced civilization using a nano-virus on a terra-formed planet goes wrong, instead leading to the rapid evolution of arthropods, and in particular, spiders. Meanwhile, after major war heralds a new dark age, the last of humanity leaves a poisoned Earth behind in order to find a new home.
Mathematics becomes a major cornerstone of the spiders' society, even taking on a religious aspect. Also, we get to see the development of technology in the spiders.
Thanks to Saud Molaib for bringing this book to my attention.
I have now had a chance to read it, and unfortunately do not agree with the assessment that it "heavily features mathematics".
The human scientist's consciousness lives on (to some extent) in digital form on an artificial satellite orbiting the planet that was her science project.
She initially uses mathematics to communicate with the creatures who are evolving there. Presumably, this is both because math could lead the recipients of the messages to more quickly develop intellectually and because it can form a first common language. And, as Saud Molaib says, their interpretation of these messages takes on a religious quality, as the primitive arachnids realize that the signals are coming from a light they see in the night sky and that the sender had something to do with their own creation.
However, not much is said or done with these mathematical/religious ideas. They are only briefly mentioned, mostly in the first half of the book, and they never really play a significant role. So, in my opinion, this nice work of science fiction only barely qualifies as mathematical fiction.
Two other ideas in the book which are only tangentially mathematical may also be of interest to readers of this website: Rather than developing electronic devices (computers, digital cameras, etc.), the intelligent spiders learn to "build" essentially equivalent devices out of ant colonies which they control chemically. Also, it is pointed out that the conflict between the two races is a manifestation of the Prisoner's Dilemma, with the main focus being on the difference in the way the situation is perceived by arachnids as opposed to humans.