Navigating Numerical Lands
Its just intuition, so it might not be feasible. Products that we use everyday, ones that have had great impact on a lot of other products, these products were built with intuition. Steve Jobs used intuition to build the foundation of Apple, but still it might not reflect onto mathematics... or so they say. Geoffrey Hinton, a man that had an impact on the AI world that we see today (i do not like him or support his ideas!) was once a failure. The man failed 3 types during his college years, yet he was still able to have a great impact on one of the world's most important industries. It is wildly believed that he lead his research through intuition, research that laid ground work and allowed future researchers to build what 100m users currently use, was explored through intuition. These are two examples, some might not realize the impact and some will.
Intuition is a great tool if used right. I've seen the fruition of using intuition the correct way. Laying the ground work for our research was a 22yr that had no advanced knowledge in mathematics or AI. Yet, he envisioned architectures and methods of high value solely through visualization and intuition. He later one found out that mankind has already explored that which he was imagining, and this allowed him to push the boundaries. A sudden discovery, is that intuition requires fundamental knowledge. Meaning, for one to utilize intuition one must have prior knowledge in what that tool is being used on. This allows the mind to explore and imagine, the mind must have a way to represent that idea it wishes to explore.
Intuition Intuition, Soo What?! I propose this, what if we explore how models like the transformers work through the numerical world. What if we study the numbers and how they change. I believe that we will capture phases that are shifted in different variations of trained models based on the position and index of the training examples. This imagined space and Fourier Transform might allow us to explore that numerical world.
Its just a proposal, but what do it know right?