Chris Rowen, the CTO of the Cadence IP group, wrapped up the recent seminar in Las Vegas (see Breakfast Bytes for the last three days) by giving a vision for the future of neural networks and deep learning. The technology is advancing fast but there are problems. One of the biggest is around training data: if programming is becoming training (at least in this area), then the training data gets more valuable. It reminds me of something Atiq Raza said to me years ago when AMD acquired his x86 design company (I forget the name). The real value in the company was that they had a really good x86 verification suite, not so much that they had a not-so-good x86 implementation. Training only works if there are large, relevant datasets. The data needs to be labeled (or else it can’t be used for training). The German traffic sign benchmark that has been widely used is obviously a somewhat artificial example, and is especially easy to label since there really isn’t any ambiguity about what the sign is, as opposed to, say, classifying X-rays as to whether they contain a tumor or not, or deciding whether an email leaving a company contains confidential information. Open datasets might become the new open source, but one can also see big advantages for companies who have valuable data to use it for competitive advantage. We know from anecdotal data that a surprisingly small amount of information can reveal a lot about our habits, finances, friends, and more. Large scale data collection for a perfectly good reason, such as traffic congestion control, may end up collecting a lot of personally sensitive data (by-catch). For example, Waze uses the information from how fast cell-phones are moving as one way to detect congestion, but it also means that it knows where we all are, and if it kept the data (I don’t believe it does), where we go regularly, how often we break the speed limit, and more. Any of these things could become big issues. Today, neural networks are largely about identification (“this is a stop sign”), but in the future it will need to be about more complex patterns and actions. I think it was at DAC, NVIDIA showed an amazing video of a car with neural networks learning to drive simply by having a human driver correct it when it went wrong. It started off driving straight off the road, of course, but later after some time (I forget how long) it was doing a pretty good job of driving normally. It’s not that different from how we teach our teenagers to drive. Chris finished with a couple of lists of things that are not surprising and then some things he thought were more controversial. His list of unsurprising things was: Neural networks will continue to proliferate in cloud-based applications Neural networks will expand rapidly into real-time embedded functions Power constraints and extreme throughput needs will drive CNN optimization in processor platforms—embedded and server Real-time neural networks evolve from object recognition to action recognition Vision-based problems dominate the computations and the high-profile deployments Expect a mad—sometimes unguided—scramble for expertise, data, and applications Finally, the more surprising and debatable list: >100X energy and >20X in bandwidth from network AND engine architecture optimization near-term In time: deployment of 1000 tera-MAC (peta-MAC) embedded, 1,000,000 tera-MAC (exa-MAC) server neural networks Network optimization evolves from ad hoc exploration to automated “synthesis”—a new kind of EDA New value chains emerge—and swing between vertical integration and disintegration—new kinds of IP, tools and data services Data is king—access to large, diverse training sets makes new winners Potential backlash over privacy and “rise of the machines” His last point was a warning that as we have more and more automation, there could be a backlash. This could come about because of privacy concerns (or a significant privacy breach). Or it could be that as our robot overlords take over more and more of what today is done by individual workers, that there could be a backlash about the rise of the machines. One thing that came up several times during the seminar was that CNNs are already better than humans at recognizing traffic signs. Well, that is somewhat academic since nobody has a “job” of recognizing traffic signs. But my understanding is that they are better at reading X-rays, doing routine legal discovery, and other tasks that until recently were beyond the reach of automation. In The Graduate , the Dustin Hoffman character is famously given the one word of career advice “plastic.” Today “data scientist” would be a pretty good recommendation to a graduate with a technical background. Previous: Hierarchical Neural Networks![]()
![]()
↧