Certificate Authentication

Baseline Training : Generative AI with Large Language Models (LLMs):

Transformer Architecture and LLMs

● What is a transformer and how does it work?

o Historical Background
o Applications in AI Today
o Architecture Overview

● Main components and features of a transformer
o Embedding and Positional Encoding
o Multi-Head Attention
o Feed Forward Neural Network
o Normalization and Residual Connections

● Transformer Models
o Self Attention ModelMulti-Head Attention
o Multi-Head Attention
o Encoder-Decoder Architecture
o Positional Embeddings
o Popular Models:
▪ BERT (Bidirectional Encoder Representations from Transformers)
▪ GPT (Generative Pretrained Encoder)

● Performance optimization and Pitfalls of transformers
o What is context length
o Mamba and Sate-Space models
o Flash attention: memory efficient, fast attention
o Sparse transformers
o Thinking like Transformers: understanding the inner workings of the transformer architecture.
o Vision Transformers
o Why we need Quantization
o Hands on tutorial: Quantize your model (GGUF)

● Improving transformers
o Retrieval augmented text generation.
o Mixture of models

Tree of thoughts

● Fine tuning
o Theory of Low-Rank Adaptation
o Hands on tutorial: Finetuning with QLora Scaling Laws and Optimization

● What are scaling laws and why are they important for LLMs?
o Over of Scaling Model
o Data and Model Size Scaling
o Computational scaling
o Parameter Efficiency Scaling

● How do scaling laws relate to the model size, data size, compute budget, and inference requirements?
● How can scaling laws help optimize the performance and efficiency of LLMs?
o Over of Scaling Model
o Data and Model Size Scaling
Training and Fine-Tuning LLMs

● Main steps and challenges of training LLMs from scratch
o Data Acquisition and maintaining
o Large Scale of needed Data, CPU and Memory.
o Optimization Challenges growing with number of parameters

● Benefits and drawbacks of fine-tuning LLMs for specific tasks
● Best practices and tools for training and fine-tuning LLM

● Landscape of open-source LLMs

Baseline Training : Fundamentals of Reinforcement Learning Introduction

● Learning through positive reinforcement
o Definition and Core Concepts
o Markov Decision Process - Dynamic Programming - Monte Carlo
o Temporal Difference Learning
o Deep Q Networks (DQN)
o Proximal Policy Optimization (PPO)
o Hands-On Implementation

Elements of Reinforcement Learning

Important Terms (Actions, States, Rewards, Policy, Value, Q-Value, etc.)

How reinforcement learning is used in LLMs & Reinforcement Learning with Human Feedback

Alternatives to RLHF: Direct Preference Optimization