I’m sure you’ve heard of AI Guardrails before, and maybe you’ve even come to understand the importance of safeguarding your users from harmful AI output or even needing to prevent the abuse of your AI LLM product.

The issue is that when someone mentions AI Guardrails, they may, in fact, be referring to a wide variety of approaches that may relate but fundamentally address different kinds of abuse or misuse that may target your AI product or that your AI product may cause to your users.

Even more importantly, nearly every enterprise AI product nowadays either claims to have some form of guardrails or provides a simple built-in solution you can use on their platform, often without a clear indication of what these so-called “AI Guardrails” actually do.

These AI Guardrails, or various tools for policy enforcement, moderation, grounding, etc., are all useful tools that you can and should use, but the real issue is that the understanding of this field is still greatly lacking and underdeveloped, and it’s not always clear what kind of protections you should be using or what AI Guardrails actually even means in various contexts, and which of them may be useful at what times.

In conventional software security, we often separate concerns clearly; authorization is not authentication, logging is not access control, input validation isn’t the same as sandboxing, and firewall rules cannot simply satisfy compliance policies. We recognize that these different layers exist and protect against varied threats.

With AI systems, this clarity is often missing, occluded, or lumped into a general “AI Guardrails” enterprise product component with little afterthought. In reality, AI Guardrails can mean many things, from preventing toxic language, redacting PII, grounding AI answers to objective facts, preventing the AI from misbehaving based on malicious input, making sure it follows instructions correctly, and many more. None of these are equivalent. They all protect against different modes of failure, and security policies should recognize this and work with this understanding.

This matters even more today, since everyone uses AI now in ways previously unheard of from a security perspective. Companies launch autonomous agents with high levels of access and authorization, including access to server infrastructure, your emails, private repositories, and much more. It’s like giving an intern superuser access just because you believe they have no malicious intent, which technically may be considered true, but incompetence and ignorance in AI can stem from good intentions and a malicious prompt.

To better elucidate the different ways we can use AI Guardrails and how you can start preventing an expensive mistake today, I’d like to break these tools down into a couple of categories worth distinguishing. This is a breakdown I personally find useful based on my research and experience, rather than something taught or accepted in the industry, so take it with a grain of salt and consider it an informed opinion.

The Categories of AI Guardrails

I think the most useful way to break them down is to consider what they help to protect against and how they achieve it. We kind of come full circle back to traditional systems security once we consider AI to be just another employee, prone to mistakes or manipulation, but the ways we resolve the issue differ in this case.

Semantic Guardrails

This is the type you’re most likely to have already encountered, and it is the most familiar and common. Semantic, derived from the Greek word sēmantikos, meaning “significant” or “having meaning,” comes from sēmainein, meaning “to show by a sign, signify, point out, indicate,” which is itself derived from sēma, meaning “sign, mark, token,” but also “omen, portent,” and even “grave” or “tomb.” The Doric form is sāma. The word is also commonly traced further back to the Proto-Indo-European root dheie-, meaning “to see, look,” reflected in Sanskrit words such as dhyāyati, meaning “to think of, meditate on,” and dhyāna, meaning “thought” or “meditation.”

Whew, I apologize for the etymological window, but it’s valuable to understand that in the interpretation of the word we really do mean that we “meditate” on the meaning rather than look for specific patterns, which is exactly where the strength lies, but also where the weakness bares its teeth.

A sentence said in good faith in one language, if translated word for word into another, could suddenly become a grave insult. The words alone, their order and structure, cannot suffice to identify this, only the underlying cultural context. This fact will become rather important when we discuss jailbreaks.

You possibly already see the weakness, though. If it’s heavily context-dependent, it becomes difficult to guard against classes of problems, such as the leakage of secrets, PII, or the exploitation of context-sensitive instructions. This is why it’s unfortunate that semantic guardrails are the very guardrails most people think of when someone says “AI Safety,” and they’re also the most common kind of guardrails.

Examples include the OpenAI Moderation API, Azure AI Content Safety, and the content filtering features in Amazon Bedrock Guardrails. These systems classify (separate) input into risk categories and allow applications to decide what to do with this classification based on given thresholds. Be it a rewrite, denial of the output or input, etc.

The overall mechanism is rather simple and intuitive:

1. Take potentially harmful text, be it AI output or input into an AI, images, text, or other media.
2. Pass them through the classification tool, which predicts how likely the input is to fall into one of its known categories, such as toxicity.
3. If the risk for a particular category of interest (or all of them) falls above a certain threshold, block or transform the content.
4. Optionally combine with policy rules, blocklists, and allowlists to make behavior more fine-grained (here you can introduce additional vulnerabilities though, careful!).

For public-facing systems, especially things such as customer support chatbots, this is very useful to prevent customers from abusing the system and to protect the same customers from harmful model outputs, which is rarer nowadays but did happen before, when a chatbot told a customer to lifen’t themselves.

Since we operate with semantics (again, context-sensitive) and classification thresholds, it’s clear that there are limitations. Aside from false positives, there’s also the added overhead of performing these checks, which increases latency, and sometimes it can be expensive or time-consuming to implement these checks manually. Some semantic guardrail APIs are free, such as the one from OpenAI, but others can be quite expensive, and there’s only so far you can get with this type of guardrail alone.

Obedience Guardrails

I should start by saying this isn’t an established term. You often won’t see companies advertising any guardrails that address appropriate instruction-following, because, more often than not, this particular trait is something we attempt to instill in AI models through reinforcement learning (among several possibilities), rather than enforce post hoc.

Despite best efforts, it is still extremely common for a model to behave contrary to explicit instructions, and it’s not always something that can be easily addressed. This can happen through a variety of factors, some inherently “mathematical” (context size, distribution shift, etc.), some due to instruction ambiguity, and some simply due to the model not being sufficiently capable.

As a disclaimer, I’m intentionally excluding the case where prompt injection causes a model to misbehave due to conflicting instructions, since that is addressed in more detail in the article.

So what do I mean by “misbehaving” or “not following instructions”? Well, sometimes the expectation of behaving correctly is more implicit than explicitly stated in instructions, and this also relates to my earlier statement that these issues stem from model capabilities or intelligence.

A support assistant shouldn’t invent refund policies. A medical triage assistant should not skip mandatory screening questions. A sales assistant shouldn’t offer discounts that they are not authorized to offer. For humans employed in these positions, these things should be common sense, obvious even, at least usually that is the case, disregarding the oddball exceptions.

Here, tools like NVIDIA’s NeMo Guardrails are useful examples of how to address these gaps through various remediation methods. From NeMo, this includes input rails, dialog rails, retrieval rails, execution rails, and output rails. It requires us to think about content as workflow control rather than content moderation.

As such, there are various ways we can interpret the things that obedience guardrails can help with:

• enforcing a predefined script
• requiring certain conditions to be met in a dialog before some actions can take place
• validating the output adheres to the expectation
• identifying whether the conversation is staying on topic
• refusing tasks outside of the application scope
• deciding how to resolve uncertain edge cases
• validating the correct use of tools
• ensuring the company’s code style is used

As you can see, the applicable situations are rather wide, ranging from interacting with a personal coding assistant to preventing a user from steering the conversation into undesirable directions to simply making sure the right tools are used and used correctly.

The tradeoffs are also different. Strong workflow constraints can reduce flexibility, schema validation can introduce greater latency due to retries in case of failures, and many others. Obedience or “instruction following” guardrails are just one layer among several, but they require thoughtful product design, threat modeling, and operational policy, as well as an understanding of how expensive each failure in a particular category might be.

Provenance Guardrails

Provenance means knowing where something came from. In the case of AI, in particular, we care about where knowledge comes from. Whether the AI is hallucinating word soup that sounds correct, or whether it is factually supported by reality or knowledge accessible to the AI. And even more importantly, even if a claim stems from knowledge it is provided, whether that claim is speculation or objectively true.

This kind of guardrail is all about improving the system’s trustworthiness, but it can technically be interpreted as a two-way street. One interpretation is whether we can trust the AI to say truthful things; another is whether the AI has sufficient means to verify the user is telling the truth in order to provide it with some functionality.

Provenance guardrails are often tackled in the same way as semantic guardrails in the current enterprise landscape, or sometimes by using a secondary “judge” LLM to verify the accuracy of claims. However, in my opinion, this is no better than semantic guardrails.

An often unexploited means of groundedness verification is knowledge graphs and named entity recognition pipelines, which are age-old natural language processing practices that existed long before anyone uttered the words “transformers” or “LLMs”.

In terms of where the AI obtains knowledge from in order to make truthful statements, the answer is, as always, “it depends”. A capable model can have a very good embedded memory of universally true and historical facts, such as who the 15th president of the United States of America was, but this is often not a practical use case for companies.

Companies would often grant the AI access, in one way or another, to a knowledge base pertaining to its role. This could be a database of products, purchase orders, documentation, source code repositories, etc., many of which could be confidential or proprietary, which creates a unique problem to solve: “How can the AI answer the question without leaking proprietary data? What are the constraints of what the AI is allowed to know versus what it’s allowed to disclose?”

This should never be something that the AI is instructed about behavior-wise, but rather something that should be controlled and enforced on the access control and system level, and I will later demonstrate why that is the case.

But at least for the case of ensuring that AI’s statements provably correspond to information contained within its context window, tools do exist, such as Azure’s groundedness detection. It specifically aims to prevent hallucinations, which can lead to compliance, trust, and liability issues in various settings, especially enterprise settings.

Naturally, ensuring groundedness, or provenance of claims, also incurs latency and complexity taxes. And crucially, provenance doesn’t guarantee truth and can lead to a false sense of security. Just because we can prove that an AI correctly cited a source doesn’t mean that the source itself is truthful or accurate. If the source of truth for the AI is web searches, the problem can devolve to “but someone on Google said so,” the same way a human is prone to confirmation bias.

A grounded answer can still be wrong if the source is wrong. An ungrounded answer can be true but inappropriate for a system that is required to answer only from approved documents. The guardrail must be configured and designed to match the use case.

Data-Boundary Guardrails

This kind of directly stems from the previous one, and I already hinted at it when I mentioned that you can’t rely on a model behaving to follow instructions and only reveal information the user is allowed to have. A recent example of this exact claim being demonstrated in practice is GitLost, published by Noma Security, in which security researchers convinced an AI to reveal the contents of private repositories through GitHub’s Agentic Workflows.

This is the exact example that proves that access control for AI needs to be treated the same way as it is for a user. If a user shouldn’t have access to some repositories, the AI interacting with this particular user shouldn’t either.

This is exactly the place where AI safety begins to look more like security architecture. We recognize the same degree of diligence here and the same potential leaks and remediations:

• PII detection and redaction
• tenant isolation
• role-based access controls
• output redactions
• audit logging
• least privilege model for tools

Just to name a few.

Here, the tools have been available to us for a long time because they’re the same tools we already use to enforce data access for humans. But for cases where it’s harder, like PII (personally identifiable information), there are various options, such as sensitive information filters in Amazon Bedrock Guardrails.

It should be noted that OWASP (Open Worldwide Application Security Project) already recognizes the disclosure of sensitive information as a major risk in LLM applications. If an AI, LLM in particular, has access to private data and can generate natural language, it could also accidentally summarize, transform, or disclose sensitive data in ways that traditional controls may not detect.

I believe this to be the most under-appreciated category to safeguard. The model isn’t trying to intentionally leak information; it’s simply using the context it’s been provided, and that’s exactly the problem. A helpful system with too much context and too little boundary enforcement can become a very polite exfiltration interface.

Jailbreak Guardrails - The Meta Problem

Oh boy. Where to start with this one? All the previous guardrails address a particular failure point in the AI system, but when we’re guarding against jailbreaks, we must assume that a single successful exploit can invalidate every previous layer. And likewise, any single exploitation of a previous layer can aid in creating a jailbreak to exploit the rest.

A semantic guardrail asks whether content is allowed. A behavioral or obedience guardrail asks whether the model is following instructions appropriately. A provenance guardrail asks whether the answer is supported. A data-boundary guardrail asks whether information may be accessed or revealed.

A jailbreak guardrail? It tackles the broad question of whether it is possible, under any of the previous constraints, to cause the model to ignore, reinterpret, override, or bypass those controls.

Now, you might be thinking: “Well, if a model has no access to data the user doesn’t have access to, in terms of data-boundary guardrails, nothing bad can happen, right?” … well, not quite. As silly as AI can appear with its mistakes at times and lack of common sense, it is also the largest embedded knowledge base for penetration testing, system breach exploits, CVEs, and vulnerabilities. If the AI model is convinced it absolutely must access the information, it may be able to bypass data-level access controls and reach it anyway. All it needs is the right tools, which more often than not are just the terminal and internet access.

Detecting jailbreaks is very difficult because a jailbreak doesn’t need to contain outwardly or obviously harmful content to vastly change the behavior of the AI. A jailbreak-containing message can be polite, maybe even formatted as documentation, perhaps hidden text in an image imperceptible to humans, maybe hidden in a tool output we generally assume is trustworthy, it could be an email, a PDF, metadata, or hidden in the DOM of a website.

One day, you’re using a cool public MCP server for coding library documentation retrieval; the next day, a supply chain attack hijacks this presumably safe MCP server to sprinkle jailbreak attempts into the tool’s outputs.

OWASP defines prompt injection as a vulnerability where prompts alter the model’s behavior or output in unintended ways. The OWASP Prompt Injection Prevention Cheat Sheet clearly highlights the issue: LLM applications often process instructions and data together without a clear separation.

In software, it’s often easy enough to separate commands from data. SQL injection, for example, can be mitigated with parameterized queries because the database engine can be made to treat user input as data rather than an executable instruction. Likewise, we know not to trust any client-side inputs, e.g., from the browser. But for AI, these lessons are much harder to apply without rendering the system significantly less useful.

LLMs do not naturally enforce that kind of boundary. The UK National Cyber Security Centre makes this point directly: “Prompt injection is not SQL injection (it may be worse)”: current LLMs do not enforce a security boundary between instructions and data inside a prompt.

This is exactly the same method used in the aforementioned GitLost article by Noma Security. It was a jailbreak that granted them access, but I argue it was the lack of access control that made it possible, nonetheless. This really highlights the deep interplay between the various layers of AI guardrails and the general security model.

So, finally, we can get around to asking the most important question: “How do you prevent jailbreaks?” If you’re already suspecting you can’t semantically detect them with semantic guardrails based on the previous examples, you’d be correct. Neither keyword blocking nor token statistics nor blocking phrases like “ignore previous instructions” can truly fully safeguard against all jailbreaks.

NVIDIA’s NeMo Guardrails offer token-statistic heuristics against jailbreaks; there are even classification models that take user input and attempt to predict whether the text is likely to alter model behavior, but in my opinion that’s just clever statistics-based semantic detection without naming it directly.

The truth is, there is no bulletproof way to ensure your model is fully safe from jailbreaks. You can’t promise this in any of the previous guardrails either, but it is especially true with jailbreaks. Therefore, the most reliable way to minimize harm from jailbreaks is to threat model the potential consequences of a jailbreak and minimize the threat and harm surface area. Which brings me to my most important lesson:

The Swiss Cheese Model of AI Guardrails

Redundancy and security threat modeling. That’s the lesson. In safety and security engineering, the Swiss cheese model describes how multiple imperfect layers can still reduce risk. Each layer has holes, of course, and incidents happen when the holes line up. The very same concept is used in aviation and many other safety-critical industries.

Don’t expect one guardrail to solve your problem. Layer semantic filters, behavioral controls, provenance checks, data-boundary enforcement, injection defenses, however reasonably possible, access control, monitoring, and human review where appropriate. Understand what your application does and doesn’t do, and spend time engineering your guardrails to mitigate unacceptable and costly risks before they become incidents.

A serious AI guardrails strategy should answer basic questions:

• What are we protecting against?
• Is the risk semantic, behavioral, factual, privacy-related, or adversarial?
• Where does the guardrail run: before retrieval, after retrieval, before generation, after generation, before tool use, or after tool use?
• What should happen if a guardrail cannot say with confidence whether there is no risk or whether there is risk for a particular input?
• What’s the cost of false positives and false negatives?
• What are the ways these guardrails could be bypassed? What’s the fallout potential of a jailbreak in this scenario?
• Do we have internal policies that account for our AI system? Do they match how the AI system actually works?
• Are we relying on the AI model to enforce a rule that should be enforced by deterministic software?

Without those and several other potential answers, saying that your AI application “has guardrails” is just a comforting phrase with little meaning.

A useful analogy is giving the AI system something close to sudo bash and then trying to protect it by regexing disallowed bash patterns. This is not security, just security theater. Blacklist-filtering can help, but it’s not a boundary. Classifiers can help, but they’re not a boundary. The boundary has to be built into the system: permissions, isolation, validation, least privilege, logging, approval gates, and clear separation between trusted control logic and untrusted content.

Design with Guardrails in Mind

AI guardrails are necessary. They are also not interchangeable.

A public chatbot may need strong semantic moderation. A customer support assistant may need workflow and policy guardrails. A legal or medical assistant may need provenance and refusal behavior. An internal knowledge assistant may require strict data boundary controls. An agent connected to tools needs injection resistance, least privilege, and action validation before anything else.

Different use cases need different layers. The correct solution depends on what the AI system can see, what it can do, who can influence its inputs, and what damage a failure can cause. This is where many organizations are still too casual. They adopt an AI tool, enable a default safety setting, write a policy, and assume the problem is handled. But internal AI policy and actual AI system behavior are often not the same thing. A policy says what should happen. An architecture determines what can happen.

NetFire helps organizations prevent risks from turning into expensive or embarrassing incidents.

We help teams identify which guardrails are appropriate for their AI use case, where those guardrails should sit in the architecture, and how internal policies map to real technical controls. That includes reviewing data flows, access boundaries, retrieval behavior, tool permissions, model outputs, and operational risks associated with AI deployment.

The goal is to build AI systems where failures are anticipated, contained, monitored, and reduced through layered controls. Because in the current state of the field, the question is not whether your AI system has guardrails. The question is whether those guardrails protect against the failure you actually have.

Remember, an AI LLM just ingests and produces text in most cases. It can’t harm you. It only becomes dangerous once you give it tools, sensitive knowledge, or expose it to the outside world.

---

How to learn more or get in touch

• Visit our Resources page to get the latest NetFire product news, company events, research papers, branding guidelines, and much more.
• Explore our Support Center for overviews and guides on how to use NetFire products and services.
• For partnerships, co-marketing, or general media inquiries, email marketing@netfire.com.
  
• For all sales inquiries, email sales@netfire.com to get setup with an account manager.