0:02 Teaching AI is really hard. Teaching
0:04 advanced prompting is even harder. This
0:06 video will make it easier. My goal is to
0:09 equip you with an understanding of the
0:10 mental models, the principles that
0:12 advanced prompters use. We're going to
0:14 go beyond the level of a prompt. I don't
0:17 have a specific magical prompt to give
0:19 you here. I will have lots of examples
0:21 in the writeup, but my goal with this
0:24 video is to actually lay out what are
0:26 the principles that advanced prompters
0:29 use that are not widely known, right?
0:32 that that you might not be aware of but
0:33 that underly a lot of these advanced
0:35 prompting techniques. Let's dive into
0:38 the first one. Advanced prompters build
0:40 self-correction systems. And so the
0:41 first category here that we're talking
0:44 about is about how you force models to
0:46 attack their own outputs and to get past
0:49 the fundamental limitation of single
0:52 pass generation. which is a fancy way of
0:55 saying you want to push the model to get
0:57 past the initial generation step into
0:59 thinking about what it's done. One way
1:01 to do this is called chain of
1:04 verification where you in the prompt
1:08 require a verification loop inside the
1:10 same conversational turn. So this might
1:12 look like you know analyze this
1:14 acquisition agreement list your three
1:16 most important findings that's not
1:19 special. Now, same prompt. Identify
1:20 three ways your analysis might be
1:23 incomplete. For each, cite the specific
1:25 language that confirms or refutes the
1:27 concern and then revise your findings
1:28 based on all of this thinking or this
1:31 verification. That's a very easy example
1:32 of chain of verification. It can
1:34 obviously get more complex. The key
1:36 thing to realize is you're not asking
1:38 the model to be more careful. That's too
1:41 vague. You're structuring the generation
1:44 process to include self-critique as a
1:47 mandatory step. and that advocate
1:48 activates verification patterns that the
1:50 model was trained on but you probably
1:52 wouldn't have gotten into by default.
1:54 Another technique in this same bucket is
1:56 adversarial prompting. And so if chain
1:59 of verification asks models to verify
2:02 their work, adversarial prompting is a
2:04 lot more aggressive. It demands that a
2:06 model finds problems even it if it needs
2:09 to stretch. And so use this when you
2:11 really really need to be sure. A good
2:13 example of this would be you need to be
2:15 sure that your security architecture
2:17 review is as complete as possible. So
2:19 you might say something like please
2:21 attack your previous design. You need to
2:23 identify five specific ways it could be
2:26 compromised. For each vulnerability you
2:27 need to assess likelihood. You need to
2:31 assess impact etc. These these
2:34 approaches are designed to be tools that
2:36 you use in specific situations. And
2:37 that's why I'm giving you these short
2:39 examples is I want to give you a sense
2:41 of not just the mental model, not just
2:43 the principle, but where advanced
2:45 prompters tend to use these techniques.
2:47 Let's go to the next one. Strategic
2:50 edgecase learning. Let's say you're
2:52 having trouble distinguishing edge cases
2:54 or boundary conditions around a
2:56 particular problem set and you're trying
2:58 to describe it in words and it's not
3:01 working well. One of the ways to handle
3:05 this is called fshot examples or fshot
3:06 prompting. And what you're trying to do
3:09 is you're trying to include examples of
3:11 common failure modes and boundary cases
3:15 so that you can teach the model how to
3:16 distinguish those gray spaces, those
3:19 edges in the situation. And so one
3:22 example of how to do this is to show
3:24 let's say you're trying to prevent a SQL
3:25 injection attack, right? The first
3:28 example might be a very obvious
3:29 injection with a raw string
3:32 concatenation. It's a baseline. The
3:34 model should pick this up. The second
3:37 example might be a parameterized query
3:39 that looks safe but has some sort of
3:42 second order injection stored somewhere,
3:45 right? Maybe stored in XSS, etc. The
3:48 failure mode would fool a naive
3:49 analysis, but you're trying to teach the
3:52 model through the example that this is
3:54 the kind of thing to look for in this
3:56 edge case. And so by including examples
3:57 of subtle failure modes, it doesn't have
3:59 to be SQL, right? I picked SQL because
4:01 that's a really interesting one. But you
4:02 can really use subtle failure modes for
4:04 much less technical subjects as well.
4:06 The model learns to distinguish what
4:08 looks secure from what is secure or what
4:10 looks correct from what is correct. And
4:12 that enables you to drop your false
4:14 negatives when you're trying to use the
4:15 model for this kind of correct
4:17 categorization work really
4:19 significantly. So those particular sort
4:21 of principles, edge case learning,
4:23 adversarial prompting, chain of
4:24 verification, those are all around sort
4:26 of how we build selfcorrection systems.
4:28 That's the larger bucket these fall
4:30 into. Advanced prompters also do meta
4:32 prompting. And I've talked about this
4:35 before. It's worth talking about again.
4:38 People do not realize how powerful meta
4:40 prompting is until they try it. And I
4:42 want to give you a couple of specific
4:44 techniques that advanced prompters use
4:46 that you can try as well. The first one
4:48 is reverse prompting. So this technique
4:50 exploits the model's meta knowledge
4:52 about what makes prompts effective. The
4:53 model's been trained on a lot of prompt
4:55 engineering conversation and you can ask
4:58 it to design an optimal prompt. And the
4:59 funny thing is people don't realize
5:02 this. You can ask it to define the
5:04 prompt to solve a particular defined
5:06 task and it will just write its own
5:08 prompt and execute on it. And so one of
5:10 the ways that you can do this is to say
5:12 you're an expert prompt designer. Please
5:15 design the single most effective prompt
5:17 to analyze quarterly earnings reports
5:19 for early warning signs of financial
5:21 distress. Consider what details matter,
5:23 what output format is most actionable,
5:25 what reasoning steps are essential, then
5:28 execute that prompt on this particular
5:31 route. You see how we've included a
5:35 request for specific outputs along with
5:37 a request for the prompt. You can
5:39 totally do that. You can ask for a
5:41 prompt with certain characteristics and
5:44 the model can use best practice, look at
5:46 the output, formulate the prompt, keep
5:48 in mind what you're looking for, and
5:49 then run it. People don't realize you
5:52 can do this, but it sure opens up a lot
5:53 of power in the model. Another one is
5:56 recursive prompt optimization. So, this
5:58 is a situation where you can say,
6:00 "You're a recursive prompt optimizer. My
6:02 current prompt is here. Your goal is
6:04 this. I need you to go through multiple
6:06 iterations with me, right? For version
6:08 one, just add the missing constraints.
6:10 For version two, please resolve
6:12 ambiguities. And for version three,
6:14 enhance reasoning depth. You can pick
6:16 what the versions do. But the point is
6:18 you are starting to define aspects of
6:20 the prompt you care about. You're not
6:22 saying what the new prompt will be.
6:24 That's up to the model. And you are
6:26 giving the model multiple iterations in
6:28 one pass. And so it's going back over
6:30 the prompt again and again and again.
6:34 And this can force a kind of structure
6:36 and constraint that improves the quality
6:39 of the prompt on the specific axes that
6:40 you care about. So those are a couple
6:42 techniques for metaring. Another
6:44 advanced technique that I want to call
6:46 out or advanced principle that we get
6:48 into with prompting is reasoning
6:50 scaffolds. How do you structure
6:52 prompting? How do you structure
6:54 interaction with the AI for deeper and
6:57 more comprehensive analysis? So self-cut
6:58 correction can catch mistakes. Meta
7:00 prompting can improve prompt design.
7:03 Reasoning is really controlling how the
7:05 model thinks and changing it by
7:07 providing a structure that forces
7:09 thorough analysis. One technique that is
7:11 not often practiced is deliberate over
7:14 instruction. So basic prompts and a lot
7:16 of the model training around token
7:19 optimization compress outputs. There's a
7:21 lot of be concise, summarize briefly,
7:23 etc. And when models are trained that
7:25 way, they may prematurely collapse their
7:27 reasoning chains. You may not want that.
7:30 And so one way to fight it is to append
7:32 at the end of your ask a really clear
7:35 definition of over instruction. Do not
7:37 summarize. You might say expand every
7:39 single point with implementation
7:41 details, with edge cases, with failure
7:44 modes, with historical context. You just
7:46 go on and on, right? And then say like,
7:48 I really need exhaustive depth here. I
7:50 don't need an executive summary. I don't
7:52 need conciseness. Please prioritize
7:55 completeness. The reason you do this is
7:57 because you want to expose the model's
7:59 reasoning to examine it. I want to
8:01 emphasize that this is about thinking
8:03 with the model. You're not doing this to
8:05 take what the model writes and just copy
8:07 paste it. This is one of those tools
8:09 that you use when you really want to
8:10 understand the problem space and the
8:12 model's thinking and respond back
8:14 effectively. So deliberate over
8:15 instruction is a big one. Another one is
8:17 called zeroot chain of thought
8:19 structure. And so this technique
8:21 exploits how LLMs are trained to
8:24 continue patterns. And so what you might
8:25 do if you're thinking about pushing the
8:27 model's thinking in a certain direction
8:30 is provide a template with blank steps
8:32 that triggers a chain of thought
8:34 automatically because the model's
8:36 objective immediately becomes filling
8:38 the structure you've laid out and that
8:40 would require decomposing the problem.
8:42 So let's say you're root causing a
8:45 technical issue and you have a series of
8:46 questions you know the model needs to
8:48 think through if it's going to do this
8:50 correctly. You can literally list the
8:52 question with a blank in sequence in the
8:54 order you think it should be correctly
8:55 done and the model will start to
8:57 structure its thinking around that
9:00 scaffolding. So this is really effective
9:02 for problems that are quantitative for
9:04 technical problems because it creates a
9:06 natural progression from the breakdown
9:08 of the problem to the solution. And that
9:10 makes it much easier to understand what
9:12 the model is thinking and to find out if
9:14 you're actually in the right territory
9:17 from a structure of thought perspective.
9:18 Is the model on the right track here or
9:21 not? Reference class priming is another
9:23 advanced technique here. Reference class
9:25 priming provides examples of reasoning
9:27 quality and asks the model to match that
9:29 explicit reasoning bar. So let's say
9:31 you're using the model's own best output
9:34 as a quality benchmark rather than
9:36 relying on human provided examples. LLMs
9:38 are trained to continue patterns and
9:39 when you show it an example of
9:42 highquality reasoning whether provided
9:44 by humans or another model and you ask
9:46 it to provide analysis that matches the
9:48 standard you are priming the
9:50 distribution of the model toward that
9:52 level of depth. This is really different
9:54 from sort of traditional fshot
9:55 prompting. You're not showing input
9:58 output pairs here to teach the model
9:59 what to do. But instead, you're
10:01 providing examples of quality reasoning
10:03 and asking the model to meet that bar of
10:06 quality. Without priming, outputs can
10:08 vary wildly on quality and format, and
10:10 it can be difficult to control for that
10:11 directly with a prompt. And so,
10:14 sometimes having an example can push the
10:16 model toward producing much more
10:18 consistent quality across an overall
10:20 document set. Another technique that I
10:23 want to call out is around perspective
10:24 engineering. So, if we talked about
10:26 reasoning, I gave you a few techniques
10:28 here. Perspective engineering is really
10:30 cool. Single perspective analysis will
10:32 have blind spots determined by the
10:34 model's default reasoning mode. So
10:36 advanced prompters will push the
10:37 perspective of the model to generate
10:39 competing viewpoints on different
10:41 priorities and that leads to higher
10:43 quality thinking as well. One example is
10:45 a multi-persona debate. So let's say you
10:47 want to simulate the perspective of
10:49 three different experts. You actually
10:51 can instantiate three experts. You can
10:53 just say three experts with conflicting
10:55 priorities need to debate. These are the
10:57 personas, right? Persona has priority X,
10:59 persona 2 has priority Y, persona 3 has
11:02 priority Z. They must argue for their
11:03 preference and critique the others
11:05 positions. After debate, you must
11:07 synthesize a recommendation that
11:09 addresses all of their concerns. You can
11:11 do this when you have something where
11:13 you need a vigorous debate, but you
11:15 don't know how to push the LLM to get
11:18 there. A good example is costbenefit
11:19 analysis for vendors, right? If you're
11:21 trying to figure out if you buy with a
11:23 vendor or if you do a different
11:25 approach, you can simulate that whole
11:26 conversation and it's not the same as
11:28 having the seauite talk about it, but it
11:31 is great preparation and it helps you to
11:32 expose perspectives and thinking you
11:35 might not be aware of otherwise is an
11:38 example of taking a very human
11:41 conversational technique like debate and
11:43 actually deliberately putting it into
11:45 the chat. deliberately getting the chat
11:47 to work like a human and argue back and
11:50 forth in ways that help us to learn and
11:52 make better decisions. Now, critically,
11:54 personas do need specific potentially
11:56 conflicting priorities. You can't just
11:58 vanilla instantiate them with no
12:00 conflicts and expect good results here.
12:02 Another technique that I think is really
12:04 important to call out is temperature
12:05 simulation. Now, temperature is this
12:08 idea that the model uh is more
12:10 deterministic, more focused when it's
12:12 low temperature or cold, and more
12:14 creative when it's hot. That's
12:16 traditionally controllable via the API,
12:19 but you can actually do that in the chat
12:22 indirectly. One way to do it is to have
12:23 the model roleplay at different
12:25 temperatures. So you could say, I want a
12:27 junior analyst who is uncertain and who
12:30 overexplains to look at this problem
12:32 first. I want a confident expert who is
12:34 concise and direct. That would be a
12:36 cooler temperature. Uh and then I want
12:38 you to synthesize both perspectives and
12:40 highlight where uncertainty is warranted
12:42 and where confidence is justified. We're
12:44 basically giving the model a low
12:45 temperature pass, a high temperature
12:47 pass, and asking it to synthesize. I
12:48 think that what's interesting here is
12:50 that we can take a lot of the techniques
12:52 that we see in the API and simulate them
12:54 effectively as advanced prompters in the
12:56 chat to get where we want to go. I hope
12:58 this has been helpful for you. There are
12:59 probably other advanced prompting
13:01 techniques I could get into. I don't
13:02 want to make this overwhelming, but
13:04 these are the kinds of mental models
13:06 that advanced prompters find difficult
13:08 to articulate, but they inform a lot of
13:10 productivity. They're very highly
13:11 leveraged examples. If you're interested
13:13 in diving in further, the writeup has a
13:15 lot more. It has sort of examples of the
13:16 prompts, etc. Enjoy. Good luck
13:18 prompting. It's so powerful. We should
