AI with continuous learning is still expanding the storage semiconductor market

Despite Calls for Slowing AI, Continual Learning AI Continues to Expand the Storage Semiconductor Market1/ Despite recent arguments suggesting that AI development should slow down, continual learning technologies that enable AI to constantly absorb new information are becoming a new growth driver in the storage semiconductor market.2/ Global investment bank Citi predicts that global HBM bit demand will increase by 62% next year compared to this year, reaching 75.2 billion Gb.3/ Citi cites continual learning as one of the main reasons for the surge in HBM demand and analyzes that as training and inference are repeated, not only HBM but also demand for server DRAM and enterprise SSDs will grow simultaneously.4/ Currently, most generative AI operates by pre-training before responding, and new information is not automatically added to long-term knowledge, resulting in efficiency issues.5/ With continual learning, the boundary between training and inference becomes blurred. The goal is for AI to learn new data and experiences while delivering services and to combine this with existing knowledge.6/ While continuously acquiring new information, it is also necessary to access massive amounts of existing data, which requires greater storage capacity. Besides HBM, using server DRAM and NAND-based eSSD in tandem is also becoming increasingly important.7/ Since learning also takes place during inference, bandwidth alone cannot resolve bottlenecks. Therefore, demand for near-memory computing technologies such as PIM may also increase.8/ Samsung Electronics has proposed zHBM, which directly stacks HBM on AI accelerators. SK Hynix has introduced PIM, SALT-KV, and HBF architectures in an attempt to overcome the limitations of the current GPU-HBM structure.