Clearly, the main point of the story is a social and political criticism of the hysteria which can develop around nationalism and religion. However, mathematics is used in the story in two ways: both as an explanation of how the story came to happen and also as an important part of the main metaphor. Specifically, the pan-dimensional journey is explained as a sort of "change of coordinates" on Einstein's four-manifold (what I would call Minkowski space) in which z and t are interchanged. As the story itself explains, other works of fiction often suggest that this would need to be done with some sort of electric field generator...but a change of coordinates is after all something that we just do in our minds. Moreover, at the end, the idea that the "other universe" was really ours viewed from a different angle is illustrated by the idea of conical sections. (See, for example, this animation or this website where it is illustrated that such seemingly different geometric objects as ellipses, circles, parabolas and hyperbolas can all be derived as an appropriate "slice" of the same cone.)

Towards the beginning of the story there is a remark that mathematical physicists, spending so much of their time thinking of big and abstract concepts, lose their appreciation for things on the human scale. (This same idea appears at the end of The Bishop Murder Case, published the previous year, to explain the horrific crimes committed by a mathematician.) Normally, I would view this as an insulting remark. However, it is perhaps balanced out by the fact that the mathematician in "the other universe" is wise enough not to get caught up in the hysteria precisely because of his mathematical insistence on precise definitions.

Another mathematical highlight of the story is the protagonist's "prayer" to Riemann and Lobachevsky as he is being taken to his execution.

The phrase "The Gostak distims the doshes" is actually a scholarly example from linguistics of our ability in English to understand syntax even when the precise meanings of the words are completely unknown. According to Wikipedia, the example was first developed by Andrew Ingraham in 1903 and was published in The Meaning of Meaning by Ogden and Richards in 1923, where Breuer presumably encountered it.

The story appeared in the March 1930 issue of Amazing Stories and has been reprinted many times since. The only recent printing that I know is in the new collection of Breuer stories The Man with the Strange Head.

I respectfully disagree. In many mathematical SF stories, the math does nothing more than serve as a vaguely believable explanation for otherwise fantastical elements. However, I think that in this story it serves one other important purpose. It is also part of an analogy at the end which helps us to understand how the same mathematical object (a conic surface) can look very different depending on how you look at it. In this sense, it adds to the social commentary in a much deeper and more important way that just explaining how he got to the other universe...it allows us to see that it was not another universe at all but only the same one seen differently.