sql question

Hi Experts, thanks for reading this.
I had a job interview yesterday where the technical interviewer really busted my chops. He is an expert SQL programmer and i am a VB programmer. One question he asked me was: if you have a table with a cloumn, and in that column the data rows may contain characters, numbers, or a combination, how would you select rows with only numbers in them?
I was reverting to my application development strategies of creating a function that would identify if there were a digit in the value. He told me that using a function would be far too slow on a table with 3 million rows, and that the way to do it was to select the UPPER case of the column and inner join it with a LOWER case select of the column. The resultant set would have no characters because only digits are unchanged between the cases. I felt stupid.
Here's the rub - he told me that using a function is far too slow. Aren't UPPER and LOWER functions? It seems to me that his trick has to operate on every row once for the uppercase, once for the lower case, and once for the comparison for an O of 3(n).
My suggestion is to compile a procedure that accepts a varchar input val and scans each character in the string for an ascii value in the digit range. As soon as one is found the functions returns true and the row is added to the result set. Doesn't that effectively reduce the processing involved?
I don't have any large data sets to work with, and never have worked on a table that size, but I am curious if anyone knows.

Thanks!