Is Computational Power a Barrier to Continued AI Research
I am currently researching whilst ill in bed the future of AI and the pace of change towards the next major milestone in AI development “General AI”. The questions I am asking is what are the barriers to reaching this next goal and even if it’s a worthy goal in the first place.
The first one I have come across is “computational capacity”. Is this holding back what looks like the inevitable evolution of AI?
Lets start at the beginning. The 1950s were at time where we saw the first press hype about AI and its potential emerge. During this period the popular press described computers at that time as “super-brains” that were “faster than Einstein”. If the goal was to model AI on how the human brain works (which we still have very little understanding) as the true definition of intelligence then these comments were clearly rubbish. Let’s compare the computational capabilities between computers Then, Now and the human brain.
Human Brain – The brain does not operate in FLOPS (floating point operations per second) like computers, but some scientific estimates suggest it performs between 10¹⁵ to 10¹⁷ operations per second (1 to 100 petaflops).
Computers 1950s – Early computers like the IBM 701 (1952) or the UNIVAC I (1951) could perform about 1,000 to 10,000 Flops per second.
Computers today – A modern high – end consumer PC with a CPU like the intel Core i9-13900k or an AMD Ryzen 9 7950X can process trillions of instructions per second (measured in teraflops) with the NVIDIA RTX 490 GPU exceeding 80 teraflops (TFLOPs) or 0.8 of a Peta flop. This is computational power which is readily available via the cloud and can be obtained and used in very short order via my own laptop. The worlds fastest computer EL Captain located at Lawrence Livermore National Laboratory in California USA runs at a peak performance of 2,746,000 petaflops
So, if emulation of the human brain is the approach we take to evolving AI to greater heights, then current computers currently exceed the capabilities of the Human Brain. In fact the Brain is starting to look distinctly slow in comparison. Clearly this lack of computational power was very much behind the first AI winter stretching to the year 2000s but is not a constraint now.
However, I think this conclusion misses the point. We are discovering that perhaps speed was never a real reason for the slow pace of AI research, and that the truth is speed alone is not going, to get us close to the goal of General AI. There is clearly access to computational power is revealing another fundamental problem that has obviously been around since the beginning of AI research.
Running a poorly designed algorithm on a faster computer doesn’t make the algorithm better, it just means you get the wrong answer more quickly and with more data there are more opportunities for wrong answers!. The principal effect of faster machines has been to make the time for experimentation shorter, so that research can progress more quickly. It’s not hardware that is holding AI back, it’s software. We don’t yet know how to make a machine intelligent – even if it were the size of the universe.
I think my local university AI professor needs to cancel his planned trip to look at new faster chips being made in China and instead focus on AI algorithm research both from a computer and human brain perspective.