# Automated System Prompt Optimization (OPRO)

Elevate your LLM application development process with Farsight OPRO: automated system prompt optimization. This library intelligently iterates through prompts to efficiently find the optimal one for any given LLM system built on OpenAI - the approach is based on the Google DeepMind paper  ['Large Language Models as Optimizers'](https://arxiv.org/abs/2309.03409).

Simply provide a dataset with inputs and target outputs *(min 3 pairs, though recommend at least 50)*, and Farsight OPRO will converge to the optimal prompt in one line of code. 

{% hint style="info" %}
Want to integrate quickly? Try out Farsight OPRO on a Colab notebook [here.](https://colab.research.google.com/drive/1QRdrQhgyX4bDmCBBahJg78woeZQcFVTZ#scrollTo=sfzimoOEr0_5)
{% endhint %}

### Installation[​](https://docs.confident-ai.com/docs/getting-started#installation) <a href="#installation" id="installation"></a>

Install our library by running:&#x20;

```
pip install farsight-opro
```

**Instantiation**

Begin using the SDK with the following few lines of code:

```python
# with openai credentials 
client = FarsightOPRO(openai_key="<openai_key>")

# with azure credentials
client = FarsightOPRO(
    openai_key="<azure_openai_key>",
    azure_endpoint="<azure_endpoint",
    api_version="<api_version>",
    model="<model_name>"
)
```

**Dataset Configuration**

The Farsight OPRO library requires only a dataset from the user. It expects datasets in the form of a list of dictionaries, as illustrated below:

```python
[{"input": "x", "target": "y"}, ... , {"input": "a", "target": "z"}]
```

For those users that may not have a dataset easily available in this format, we recommend generating a synthetic dataset from ChatGPT to get started. Below is a suggested prompt that will quickly generate a dataset that meets the Farsight OPRO spec:

{% code overflow="wrap" %}

```
'''Create an input-output test set in the form of a python list of dictionaries with 10 samples, reflecting question-answer pairs that might be asked during [describe your use case]. Each item in the list should be a dictionary with keys "input" and "target".'''
```

{% endcode %}

### Example Usage <a href="#installation" id="installation"></a>

```python
import json
import random
from sklearn.model_selection import train_test_split
from opro import FarsightOPRO

# replace with your openAI credentials
OPEN_AI_KEY = "<openai_key>"
farsight = FarsightOPRO(openai_key=OPEN_AI_KEY)

# load dataset
dataset_path = "/content/movie_recommendation.json"
with open(dataset_path, "r") as file:
    data = json.load(file)

# split dataset
dataset, test_set = train_test_split(
    data["examples"],
    train_size=0.4
)

##################### For a short test run, try this #####################

# dataset = [
#     {'input': 'Find a movie similar to Batman, The Mask, The Fugitive, Pretty Woman:\nOptions:\n(A) The Front Page\n(B) Maelstrom\n(C) The Lion King\n(D) Lamerica','target': '(C)'},
#     {'input': 'Find a movie similar to The Sixth Sense, The Matrix, Forrest Gump, The Shawshank Redemption:\nOptions:\n(A) Street Fighter II The Animated Movie\n(B) The Sheltering Sky\n(C) The Boy Who Could Fly\n(D) Terminator 2 Judgment Day', 'target': '(D)'},
#     {'input': "Find a movie similar to Schindler's List, Braveheart, The Silence of the Lambs, Tombstone:\nOptions:\n(A) Orlando\n(B) Guilty of Romance\n(C) Forrest Gump\n(D) All the Real Girls", 'target': '(C)'},
#  ]
# prompts_and_scores = farsight.generate_optimized_prompts(dataset, prompts_generated_per_iteration=2, num_iterations=3)
# print(prompts_and_scores)

########################################################################


# get optimized prompts
prompts_and_scores = farsight.generate_optimized_prompts(dataset, test_set)
print(prompts_and_scores) 
#  [{
#        "prompt": "Choose the movie option that aligns with the given movies' genres, popularity, critical acclaim, and overall quality to provide the most accurate and comprehensive recommendation."
#        "score": 0.94,
#        "test_score": 0.88
#
#   },
#   {
#        "prompt": "Choose the movie option that aligns with the genres, themes, popularity, critical acclaim, and overall quality of the given movies to provide the most accurate and comprehensive recommendation."
#        "score": 0.9,
#        "test_score": 0.86
#   }, ...
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://api.farsight-ai.com/farsight-opro/get-started/automated-system-prompt-optimization-opro.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.