Hello everyone, I’m Lisa. In this article, I’ll be discussing the paper and implement details of DSPy.
Manually try and error on prompt is time consuming and inefficient. To address this problem, DSPy propose a framework to automatelly generate and fine-tune prompts. All the user needs to do is assign the task and provide a few examples.
MODEL CALLS INTO SELF-IMPROVING PIPELINES
DSPy define 3 different abstraction modules and them will co-work automatically to generate or fine-tune prompt. I will explain each of them below.
Signatures and Modules
MODEL CALLS INTO SELF-IMPROVING PIPELINES
Signature is an interface that define the input and output of the task, e.g. “question -> answer” means the task is input question and then output answer.
Modules will be definded with a signature and the role of modules is implement signature and also track the intput/output trace to help teleprompter to iteration and augumentation.
DSPy provide several modules which will use difference behavior to implement the signature.
- The Predict Module: Predict Module takes in keyword arguments corresponding to the signature input fields (e.g., question), formats a prompt to implement the signature and includes the appropriate demonstrations
- ChainOfThought
- ProgramOfThought
- MultiChainComparison
- ReAct
Teleprompters and Compiler
teleprompter, which is an optimizer that takes the program, a training set, and a metric — and returns a new optimized program.
Fine-tune with small training data
teleprompter, which is an optimizer that takes the program, a training set, and a metric — and returns a new optimized program.
MODEL CALLS INTO SELF-IMPROVING PIPELINES
Stage 1: Candidate Generation
The compiler first (recursively) finds all unique Predict modules (predictors) in a program, including those nested under other modules. For each unique predictor p, the teleprompter may generate candidate values for the parameters of p:
- the instructions
- field descriptions
- demonstrations (i.e., example input–output pairs) (The most important one)
Stage 2: Parameter Optimization
Now each parameter has a discrete set of candidates: demonstrations, instructions, etc
BootstrapFewShot — Test multiple combinations of different candidates, with a primary focus on generating demonstrations. Starting from having no demonstrations at all, the process tests various combinations to determine which combination the best results.
Stage 3: Higher-Order Program Optimization
A different type of optimization that the DSPy compiler supports is modifying the control flow of the program.
One of the simplest forms of these is ensembles. An ensemble will bootstrap multiple copies of the same program, and then runs them all in parallel and reduces their predictions into one with a custom function (e.g., majority voting).
Other Tools
Tools DSPy programs may use tools, which are modules that execute computation. * retrieval models through a dspy.Retrieve module.
- Retriver: ColBERTv2, Pyserini and Pinecone
- (Future) dspy.SQL for executing SQL queries
- (Future) dspy.PythonInterpreter for executing Python code in a sandbox
Thank you for taking the time to read my article! If you found the content helpful or insightful, I’d really appreciate it if you could give me a clap～