I don't mean that programming is mathematical (which it of course is), or that mathematics is mechanical (which it sometimes also is). I mean that the actual process of software design and writing code is a lot like the process of doing and creating mathematics.
In both cases, you are trying to define the right structures so that they do the right thing. In software design, you are defining the parts and interactions of a complex system, to get it to produce the functionality desired. (If that sounds gross and OOP to you, replace "parts and interactions" with "modules and dependencies".) In mathematics, you're defining mathematical objects so that they do something interesting and trying to prove it. In both cases, it's trying to find the right definitions for the job.
One of the key differences is that mathematicians want their definitions to do something exciting and unexpected. If a piece of software does something of the sort, we usually call that a bug.
But not always. There are certain software design approaches which value the unexpected, usually by harnessing the emergence of functionality from the juxtaposition of many small modules. Like the UNIX philosophy. Or functional programming approaches, where power often comes from combinatoric explosion of definable functions. If you had a reductionist bent, you could argue that mathematics is just the exploration of the uncomputably large space of statements that may be derived from axioms and logical inference rules. Good definitions are those which make that space most accessible.
I think there is a key takeaway from this for software design. A lot of well-designed code is well-designed because it's designed with the pedantry and aesthetic sensibilities of a mathematician. A well-designed piece of code naturally covers its edge cases by the same token as a well-designed definition.