What is GenAI?

AI that's generate new content like text, image, audio and video.

Examples

Example of GenAI are GPT-3, DALL-E, OpenAI Codex, etc.

GenAI Concepts

• Tokenization: Converting raw text into tokens.

Why we should tokenize

• Tokenization convert it into a format that the model can understand.
• It creates a mapping between the raw input text and the model vocabulary.

Context window

• It's the number of tokens that the model considers when generating the next token.
• It helps the model to understand the context of the input text.
• Accuracy and relevance: The context window helps the model to generate more accurate and relevant output.
• Cost and performance: The context window affects the cost and performance of the model.

Embedding

• It's the process of converting tokens into vectors.
• Vectors have a high dimensionality to capture many features for one input token.
• Embedding models can power search applications.