Hey there, feeling lost in the ChatGPT craze? Don’t worry, you’re not alone! This powerful language model created by OpenAI is taking the world by storm and changing the way we interact with machines. It’s everywhere, from social media to TikTok, and it’s the hottest topic in technology right now. But what exactly is ChatGPT and how can it benefit you? Strap in and get ready for a wild ride as TechSparks breaks down everything you need to know about ChatGPT!
What is ChatGPT?
ChatGPT, an intelligent chatbot program, was released by OpenAI in the United States on November 30, 2022. It features natural language processing tools driven by artificial intelligence technology, and essentially serves as a pre-trained large model for NLP natural language understanding. Despite the longstanding popularity of artificial AI, nothing has compared to ChatGPT in terms of capabilities:
- It can converse with humans as if they were friends, rather than just outputting content.
- It can complete advanced tasks such as writing emails, video scripts, copywriting, translation, and coding.
- It can acknowledge mistakes and reject excessive requests, like humans.
- It can provide distinct answers by adjusting to the application scenario.
- In test situations, ChatGPT performs better than 90% of people.
Prior to the ChatGPT APP release, neural network programs could only solve one NLP natural language understanding problem or task. However, ChatGPT allows computers to understand human language by utilizing pre-trained large models, thereby completely altering the pattern of neuron network programs. In simpler terms, it involves disassembling a previous neuron network into a pre-trained large model + fine-tuning/fine-tuning method, making it easier to adapt to various NLP tasks. Users no longer need to develop a complete artificial intelligence program from scratch to meet their NLP needs. Rather, they can simply fine-tune the characteristics of the enterprise/industry on the basis of the large model, and quickly obtain an AI program that suits their needs – AI robots for enterprises/industries.
Of course, the pre-trained large model has its limitations. For instance, it cannot predict the future or everything that has happened in the near future – even journalists cannot accomplish this. Nonetheless, its power cannot be ignored. Just two months after its launch, the number of active ChatGPT users surpassed 100 million.
What makes ChatGPT APP different
AI intelligent chat robots have been a concept since the 20th century, but despite the existence of similar products, they failed to make a significant impact until the emergence of OpenAI’s ChatGPT. One of the main reasons for ChatGPT’s popularity is its focus on generative AI, which represents a new direction for AI.
Artificial intelligence is a vast and complex field that encompasses many algorithms, and one important direction of AI is machine learning based on deep neural networks. The neural network algorithm consists of “parameters,” which refer to the nodes of each layer of the network and the weight connections between them. Large models are those with tens of billions or hundreds of billions of parameters, and they are becoming increasingly common.
There are two main types of AI tasks: decision-making AI and generative AI. Decision-making AI involves answering multiple-choice questions, and before ChatGPT, it was the dominant AI task direction. Many AI applications, including Siri, Cortana, Alexa, Xiaoai, and Xiaodu, are question-answering robots. In contrast, generative AI involves creating new content, such as through chatbots. ChatGPT is a leading example of a generative AI model that can generate high-quality content, such as writing press releases.
The introduction of new algorithms, including situational learning, thinking chain, natural instruction learning, and instruction learning, has enabled ChatGPT to go beyond simple question-answering and become a true chatbot. However, training an AI model with billions of parameters is a time-consuming and computationally expensive task, which has been a significant barrier to the development of large models. Researchers and R&D personnel must carefully consider the costs and benefits of switching algorithms or adding new ones.
Concept of Generative AI
As a technician, it is important to understand the concept of generative AI and how it works. Generative AI is an artificial intelligence algorithm that utilizes existing data and content to create new and original data, often exceeding human expectations. Its potential applications are vast and varied, from creative dialogues between robots and humans to generating commercial content like meeting minutes, commercial manuscripts, and images.
Generative Adversarial Networks (GANs) were one of the early generative AI algorithms popularly used for unsupervised learning. However, the emergence of ChatGPT can be considered a significant milestone in generative AI. Unlike earlier algorithms, OpenAI is powered by large-scale pre-trained models that provide ChatGPT AI with general human knowledge and general-purpose capabilities.
One of the exciting features of ChatGPT is the Chain of Thoughts (COT) proposed by Google Scholars in 2022. The thinking chain is a series of intermediate reasoning steps that greatly enhance the model’s reasoning performance. By generating the thinking chain first, the model can significantly improve its performance without the need for fine-tuning the model parameters.
With the situational learning, natural instruction learning, and other emerging algorithms, generative AI powered by large-scale pre-trained models like ChatGPT can have vast commercial applications beyond text generation, including image and video generation. As a technician, it is important to stay up to date with these technological advancements to ensure that we can utilize them effectively and efficiently.
Relationship Between OpenAI's GPT Series Products
GPT, which stands for Generative Pre-training Transformer, is a popular natural language pre-training model. This model has several versions, including GPT-2, GPT-3, and GPT-3.5/ChatGPT, with different parameter sizes.
GPT is based on the Transformer model, which is an open-source machine translation model launched by Google in 2017. Transformer’s biggest advantage is parallel computing, which is especially suitable for distributed shared computing infrastructure like cloud computing. This model was designed to adapt to Google Cloud TPU, and this was one of Google’s original intentions. Many new algorithms have emerged on the basis of Transformer, especially laying the foundation for large models.
GPT-2, GPT-3, and GPT-3.5 are large models based on Transformer, with different parameter sizes. InstructGPT and ChatGPT are versions of GPT-3.5 that introduce human feedback and data annotators to manually fine-tune the model output. GPT-4 is expected to have multi-modal fusion capabilities, meaning it can recognize images and perform natural language processing tasks.
While OpenAI was initially an open-source company, the training and operation and maintenance costs of large models are extremely expensive, leading OpenAI to become a closed profit-making company. OpenAI has also verified the value of large models, which can perform beyond people’s imagination.
ChatGPT Operation Cost
ChatGPT APP has garnered global attention and sparked a new wave of AI entrepreneurship, but the cost of benchmarking against it is extremely high. In fact, it’s not just an AI software but a new infrastructure. For countries, ChatGPT is a strategic national resource that requires significant investment. In the future, ChatGPT and its subsequent versions may become a new competitive advantage among major powers.
So, how expensive is ChatGPT? Public information shows that the parameters of ChatGPT are as high as 175 billion, and the pre-training dataset is as high as 45TB. This is a significant leap compared to GPT-1 and GPT-2, with parameters of 117 million and 1.5 billion, and pre-training data of 5GB and 40GB, respectively. The computing power required to train ChatGPT once is as high as 3640 PFlop/s-day, and the cost of one ChatGPT training is estimated to be as high as about $4.5 million.
To run ChatGPT, a conventional data center with a computing power of 500P is not sufficient. At least 10 such data centers are needed, with an investment cost of 20-30 billion yuan. To train GPT-3, 1024 80GB A100s can reduce the duration to one month, with a cost of $150 million. A large model may require more than 30,000 A100 GPUs, with an initial investment cost of about $800 million and a daily electricity cost of $50,000. Microsoft has built a huge GPU resource pool for OpenAI in its Azure global infrastructure, consisting of thousands of GPUs.
Microsoft’s Relationship with OpenAI
Microsoft has been a long-time supporter of OpenAI, investing significant amounts of money and resources into AI research and development. In 2019, Microsoft invested $1 billion in OpenAI and promised to build a super AI computer for OpenAI that could train and run large models. This required significant modifications to hardware, network, and software, as GPUs rather than CPUs were needed for massively parallel computing. Microsoft led this attempt and continued to support chatbot progress since its inception.
Recently, Microsoft invested an additional $2 billion in OpenAI and pledged to invest another $10 billion in 2023 to support ChatGPT APP research and development. Beginning in 2023, Microsoft plans to incorporate OpenAI technology into its products and services on a large scale. This includes launching Github Copilot in 2022, followed by Dynamics 365 Copilot and Microsoft 365 Copilot in 2023, providing an enhanced AI experience to ordinary users. Microsoft 365 Copilot and New Bing will also embed GPT/ChatGPT technology into Office software and the Bing search engine, respectively, which has generated significant interest.
Moreover, Microsoft provides OpenAI services worldwide through the Azure Intelligent Cloud, including GPT and DALL-E. This investment in OpenAI demonstrates Microsoft’s commitment to advancing AI technology and using it to improve its products and services.
Emergence of large models
Large models have revolutionized the AI industry, with parameter scales reaching hundreds of billions or even trillions. Despite doubts about the value of larger model parameters, the emergence of ultra-large-scale models like ChatGPT chatbot has demonstrated the power of emergent intelligence.
Emergent intelligence refers to the ability of a large model with hundreds of billions of dynamically connected knowledge to demonstrate intelligence beyond expectations. With pre-training data primarily sourced from the internet, when a super-large-scale AI model learns all internet data, its intelligence will exhibit emergence. For instance, ChatGPT online can write professional business copy, create poetry and literature, engage in philosophical dialogues, and even write high-quality academic papers. GPT-4 has been shown to easily pass exams like the bar exam and university entrance exam, achieving full or near-full marks.
ChatGPT’s capabilities extend to programming, with Github Copilot based on GPT/ChatGPT technology launched in 2022. Millions of programmers on Github have experienced the high quality and efficiency of AI programming, with some even reporting that up to 80% of their code is automatically generated by Github Copilot. The search engine New Bing can also directly convert Python code into Rust code, demonstrating the potential for AI to revolutionize coding.
Industry experts suggest that by increasing the amount of calculation, data volume, and model parameter scale simultaneously, the performance of the model can be improved without limit. By connecting human knowledge, thinking, and memory capabilities, a miracle of evolution may emerge, with infinite improvements in the level of intelligence.
Large Models Outside OpenAI
The world of natural language processing (NLP) has seen an incredible surge in the size of model parameters. From large models with tens of billions to super-large models with hundreds of billions or even trillions, many technology companies are competing to build the largest models. However, due to the high costs involved in training and operating these models, very few commercial companies are willing to make real investments in them.
OpenAI, as a public welfare organization, received billions of dollars from Microsoft to develop ChatGPT. Since then, many companies have launched their own NLP models based on Google’s Transformer, including Microsoft’s Turing-NLG with 17 billion parameters, Google’s Switch-C with 1.6 trillion parameters, and many others.
In addition to large NLP models, many companies have also developed multi-modal models that can handle both NLP and cross-modal tasks such as image recognition and visual-language processing. Examples of these models include Ali’s M6, the Institute of Automation of the Chinese Academy of Sciences’ “Zidong Taichu,” Baidu’s Wenxin, and Google’s PaLM-E and Clip.
Although the costs of building and operating these models are high, the potential benefits in terms of natural language understanding and cross-modal tasks are immense. As a result, many technology companies continue to invest in the development of ultra-large models.
Benefit From ChatGPT Wave
The emergence of chat openai has revolutionized the way we live, work and learn. With its ability to connect knowledge and improve work efficiency, ChatGPT has substantially increased productivity in the workplace. Microsoft has released a series of products, including Copilot and New Bing, which have demonstrated how ChatGPT can enhance work quality and ability. Copilot can summarize the key points of online meetings, while Photoshop can automatically modify images based on instructions given by employees. As a result, employees no longer need to learn Photoshop or other software to perform such tasks.
ChatGPT has also made it easier for individuals to become programmers. With its ability to generate code through natural language, even those without any programming knowledge can create high-quality software and programs. This development has significant implications for the future of work and the job market.
The impact of ChatGPT is not limited to the workplace. It has also significantly improved our ability to learn and be entertained. ChatGPT has drawn the attention of the education community, with its excellent teaching ability allowing education to be more accessible to remote areas and people. However, it also puts teachers at risk of being replaced, and raises concerns about the future of education. ChatGPT can also generate homework that can easily pass tests and exams, which raises concerns about the impact on students and the education system.
In our personal lives, ChatGPT has revolutionized the way we interact with technology. It has upgraded intelligent voice assistants and customer service, while also allowing us to chat with the elderly, talk to children, and create poetry, paintings, and music. The possibilities are endless.