Stanford Launched A RAG (Retrieval-Augmented Generation) Online Course and Close Versus Open Ecosystem

The Hottest Architecture for LLM Application Development, from Here We Think about Close Versus Open Ecosystem

Image generated by DALL.E-3

Three days ago, Stanford launched a RAG (Retrieval-Augmented Generation) online course. RAG has become a popular LLM application architecture in the industry. It showcases the current situation that companies play a leading role in technology development in AI and universities follow, while the contribution of universities and other entities remains significant, and the landscape is characterized by a dynamic and collaborative ecosystem.

RAG Architecture, Image from Stanford Online

RAG Architecture, Image from Stanford Online

From a knowledge sharing perspective, there are two approaches. One is the knowledge protection system centered on patents, and the other one is a public incentive mechanism driven by papers and open-source code. The latter is cost-free and low-barrier, allowing everyone to participate in innovation. These two models are also hot topics in the current AI regulation discussion on the coexistence of each other.

For the open ecosystem, while papers, code, and free courses on platforms like GitHub, Hugging Face, and YouTube are significant, the scope is much broader and includes various other components that support collaboration, learning, and development in the tech community.

Below are major components and trends for the open ecosystem, especially in IT and Artificial Intelligence industry.

• Companies: Indeed, large technology companies are leading significant developments in IT and AI. These companies often have substantial resources, access to large datasets, and the ability to attract top talent, which positions them at the forefront of technological innovation.
• Universities: While it’s true that universities often follow industry trends, they still play a critical role in foundational research and theory. Universities are hubs for pioneering research that may not have immediate commercial applications but are vital for long-term technological advancements.
• Collaboration: There is a growing trend of collaboration between industry and academia. Companies often fund university research, and scholars frequently consult for companies. This collaboration leads to a more integrated development landscape.
• Innovation Ecosystem: The ecosystem of innovation is complex and involves more than just companies and universities. Startups, independent researchers, open-source communities, and government research labs also contribute significantly.
• Diverse Contributions: It’s important to recognize that while large companies lead in certain areas, especially in applying AI at scale, many innovative ideas and foundational research still emerge from universities and smaller entities.
• Global Perspective: The dynamics might vary globally. In some regions, universities are at the forefront of technology development, while in others, companies lead the way.
• Rapid Change: The landscape of technology development, especially in AI, is rapidly evolving. What is true this year might change in the near future as new players emerge and existing ones shift their focus.

There is a famous saying: “OpenAI is not open”, which refers that GPT-3 and GPT-4 from OpenAI are not open sourced. On the other hand, OpenAI has publish a paper for GPT-3. By the way, the ERP ecosystem is also a closed system.

I interpreted the architecture of RAG in an YouTube video on November 1 last year. I remember stumbling upon an article on Medium introducing RAG and thinking it was a great method. It indeed became very popular and is now the mainstream architecture for large language model (LLM) applications.

Another term, “Chain of Thoughts,” which I found very striking when I read about it on Medium for the first time around March last year, has now become a standard term for explaining the inference mechanism of large language models and the theoretical basis for agent application development.

#RAG #StanfordUniversity #LLM #ChainOfThoughts