This is the most famous story by Lovecraft, which spawned it's own sub-genre and RPG, called the Cthulhu Mythos. It concerns the investigations of Prof. Francis Wayland Thurston as he investigates the estate and events leading to the bizarre death of his grand-uncle Prof. George Gamell Angell of Brown University. It seems Prof. Angell's anthropological research uncovers a cult connected to a vast, alien city submerged beneath the Pacific and composed of some strange non-Euclidean geometry, where "the angles are all wrong". Entombed in this alien city is a horrid alien octopoid race and their monstrous master, Cthulhu, who are waiting for the time "when the stars are once again right" to reclaim the earth as their own.

I don't really remember much explicit math here; however, Lovecraft definitely solved the Drake equation, at least heuristically, and calculated how much time would pass between the arising of species. Then he uses this, correctly, to justify terrible incomprehensible alien invasions and magic - in a 1920's novella. it's really quite amazing. He also correctly describes the appearance of a hyperdimensional city quivering in and out of our hyperplane. He is also the only author I have read that used "centillions" to number something. It was bad, of course. It's too bad "Call of Cthulhu" isn't actually that great a story.