Farsight AI Starter Docs
  • Get Started
    • Welcome to the Farsight AI Starter Library
      • Overview
      • Getting Started
  • Metrics
    • Standard Metrics
    • Custom Metrics
  • Fully Automated Prompt Optimization
    • Introduction
      • Tutorial
    • Generation and Evaluation
  • Step by Step Prompt Optimization
    • Introduction
      • Tutorial
    • Prompt Generation
    • Prompt Evaluation
      • Rubric Development
Powered by GitBook
On this page
  1. Fully Automated Prompt Optimization

Introduction

Automatically uncover an optimized system prompt that will drive your system towards high performance

PreviousCustom MetricsNextTutorial

Last updated 1 year ago

Farsight AI utilizes the leading approach for prompt optimization by incorporating the to evaluate systems prompts and LLM prompt optimization using .

We automate the entire prompt generation and evaluation process. The system only requires shadow input traffic from your intended use case. Our prompt optimization process works by automating each of the following steps:

  1. Generation of your use case description, an evaluation rubric, an example response based on the given shadow traffic.

  2. Iterative generation of system prompts, improving each iteration by providing previous system prompts and scores.

  3. Finally, we select the few best system prompts for your convenience.

Prometheus prompt evaluation rubric
ORDO