Quels sont les différents types de modèles existants ?

Comment sont il entrainés ? Pour quel type de cas d’usages ?

Différents types de modèles

Encoder only models — Autoencoding models — Mask Language Modeling

• Trained using Mask Language Modeling
• Build bi-directionnal objective models
• Denoising objective - Reconstruct the original sentence
• Use cases
  • Sentiment analysis classification
  • NER
  • Word Classification
• Models:
  • BERT
  • ROBERTA

Decoder only models — Autoregressive models — Next token prediction

• Use cases
  • Text generation
• Models:
  • GPT
  • BLOOM
  • LLaMa

Encoder-Decoder models — Sequence To Sequence models — Span Corruption

• Mask random sequences of tokens
• Replace by sentinel token
• Unidirectional models
• Sequence-Sequence task
• Input & Output are different lenghts

Use cases:

• Transation