Like other AI components, ChatGPT interacts with users through a single-line text input field and provides text-based results. However, ChatGPT has a particularity which lies in the way it makes information available: it is a question of giving an answer based on the context and the intention of the user's question. This allows the user to ask certain questions that cannot be answered by Google, for example. Faced with this observation, one might wonder how this tool works? Discover the answer in this article, which discusses the main phases of its operation.
AI pre-training
What makes ChatGPT stand out from other AI tools is its ability to analyze queries and produce detailed responses and results based on most of the world's digitally accessible textual information. Just like the Google search engine which has a data collection phase and a user writing phase, Chat GPT includes two operating phases which are pre-training and inference.
Generally, the AI pre-trains using two main approaches, namely the supervised approach and the unsupervised one. Since the creation of this tool until today, most of the artificial intelligence system projects, it is the supervised pre-training which is the most adopted. It is indeed a process in which a model is trained on a set of labeled data, where each input is associated with a corresponding output.
For example, it's obvious that an AI is trained on a data set of customer service conversations. In the case at hand, the concerns or questions of the users are tagged with the appropriate answers from the customer service representative.
As for unsupervised pre-training, it is a process that, as the name suggests, promotes the training of a model on data where no specific output is associated with each input. This model is the most widely used formula for clustering, anomaly detection, and dimensionality reduction.
The architecture of Transformer
Transformer architecture is a type of neural network used to process natural language data. As a reminder, a neural network is intended to simulate the functioning of the human brain by using layers of interconnected nodes to process information. This phase of ChatGPT relies on ''self-attention'' to process sequences of words in order to assess the importance of different words in a sequence while making predictions.
Self-attention refers to how a reader can rely on a previous sequence or paragraph to understand the context in which a new word encountered in a book is used. The role of the Transformer is therefore to examine all the words of a sequence in order to understand the context and the relationships between them. It is made up of a number of layers which, in turn, are made up of several sub-layers. The two best-known layers are the self-attention layer and the anticipation layer.
The first is used to calculate the importance of each word in the sequence, while the second aims to apply nonlinear transformations to the input data. These layers help the Transformer learn and understand the relationships between words in a sequence. During the transformation, it receives input data, such as a sentence, which allows it to make predictions.
Similar
