Applied AI Software Engineering: RAG

Applied AI Software Engineering: RAG -

by: Gergely Orosz url: https://newsletter.pragmaticengineer.com/p/rag

Highlights

Quote

The most obvious solution is to input the additional information via a prompt; for example, by prompting “Using the following information: [input a bunch of data] please answer the question of [ask your question].” (View Highlight)

Quote

An option is to update these weight matrices based on additional information we’d like our model to know. This can be a good option, but it is a much higher upfront cost in terms of time, money, and computing resources. Also, it can only be done with access to the model’s weightings, which is not the case when you use models like ChatGPT, Anthropic, and other “closed source” models. (View Highlight)

Quote

With RAG, you control the underlying data source, meaning you can scope the LLM to only have access to a single jurisdiction’s data, which ensures responses are consistent. (View Highlight)

Quote

Here are the steps to building a RAG pipeline: Step 1: Take an inbound query and deconstruct it into relevant concepts Step 2: Collect similar concepts from your data store Step 3: Recombine these concepts with your original query to build a more relevant, authoritative answer. (View Highlight)