Generative AI Principles

Generative AI framework for brands

A baseline framework, providing practical considerations for brands planning or developing a generative AI solution


The UK government has defined ten common principles to guide the safe, responsible and effective use of generative AI in organisations. The white paper A pro-innovation approach to AI regulation, sets out five principles to guide and inform AI development in all sectors. This framework builds on those principles to create ten core principles for generative AI use in any organisation.

You can find posters on each of the ten principles for you to display in your organisation.

Principle 1: You know what generative AI is and what its limitations are

Generative AI is a specialised form of AI that can interpret and generate high-quality outputs including text and images; opening up the potential for opportunities for organisations, including delivering efficiency savings or developing new language capability. 

You actively learn about generative AI technology to gain an understanding of what it can and cannot do, how it can help and the potential risks it poses. 

Large language models lack personal experiences and emotions and don’t inherently possess contextual awareness, but some now have access to the internet. 

Generative AI tools are not guaranteed to be accurate as they are generally designed only to produce highly plausible and coherent results. This means that they can, and do, make errors. You will need to employ techniques to increase the relevance and correctness of their outputs, and have a process in place to test them. 

Principle 2: You use generative AI lawfully, ethically and responsibly

Generative AI brings specific ethical and legal considerations, and your use of generative AI tools must be responsible and lawful.

You should engage with compliance professionals, such as data protection, privacy and legal experts early in your journey. You should seek legal advice on intellectual property, equalities implications and fairness and data protection implications for your use of generative AI. 

You need to establish and communicate how you will address ethical concerns from the start, so that diverse and inclusive participation is built into the project lifecycle.

Generative AI models can process personal data so you need to consider how you protect personal data, are compliant with data protection legislation and minimise the risk of privacy intrusion from the outset.

Generative AI models are trained on large data sets, which may include biased or harmful material, as well as personal data. Biases can be introduced throughout the entire lifecycle and you need to consider testing and minimising bias in the data at all stages.

Generative AI should not be used to replace strategic decision making. 

Generative AI has hidden environmental issues that you and your organisation should understand and consider before deciding to use generative AI solutions. You should use generative AI technology only when relevant, appropriate, and proportionate, choosing the most suitable and sustainable option for your organisation’s needs.

You should also use the AI regulation white paper’s fairness principle, which states that AI systems should not undermine the legal rights of individuals and organisations. And that they should not discriminate against individuals or create unfair market outcomes.

Principle 3: You know how to keep generative AI tools secure

Generative AI tools can consume and store sensitive information and personal identifiable information if the proper assurances are not in place. When using generative AI tools, you need to be confident that your organisation’s data is held securely; that the generative AI tool can only access the parts of your organisation’s data that it needs for its task. 

You need to ensure that private or sensitive data sources are not being used to train generative AI models without the knowledge or consent of the data owner.

Generative AI tools are often hosted in places outside your organisation’s secure network. You must make sure that you understand where the data you give to a generative AI tool is processed, and that it is not stored or accessible by other organisations.

Your data can contain sensitive and personal information that must be processed lawfully, securely and fairly at all times. Your approach must comply with the data protection legislation. 

You need to build in safeguards and put technical controls in place, this includes: content filtering to detect malicious activity and validation checks to ensure responses are accurate and do not leak data.

Principle 4: You have meaningful human control at the right stage

When you use generative AI you need to make sure that there are processes for quality assurance controls which include an appropriately trained and qualified person to review your generative AI tool’s outputs and validation of all decision making that generative AI outputs have fed into. 

When you use generative AI to embed chatbot functionality into a website, or other uses where the speed of a response to a user means that a human review process is not possible, you need to be confident in the human control at other stages in the product life-cycle. You must have fully tested the product before deployment, and have robust assurance and regular checks of the live tool in place. Since it is not possible to build models that never produce unwanted or fictitious outputs (i.e. hallucinations), incorporating end-user feedback is vital. Put mechanisms into place that allow end-users to report content and trigger a human review process.

Principle 5: You understand how to manage the full generative AI lifecycle

Generative AI tools, like other technology deployments, have a full project lifecycle that you need to understand. 

You and your team must know how to choose a generative AI tool and how to set it up. You need to have the right resource in place to support day-to-day maintenance of the tool; you need to know how to update the system, and how to securely close the system down at the end of your project.

You need to understand how to monitor and can mitigate generative AI drift, bias and hallucinations. You have a robust testing and monitoring process in place to catch these problems. 

You should use the NCSC cloud security principles to build a clear understanding of cloud security.

Principle 6: You use the right tool for the job

You should ensure you select the most appropriate technology to meet your needs. 

Generative AI is good at many tasks but has a number of limitations and can be expensive to use. You should be open to solutions using generative AI as they can allow organisations to develop new or faster approaches to the delivery of services, and can provide a springboard for more creative and innovative thinking about industry sector problems. You can create more space for you and your people to problem solve by using generative AI to support time-consuming administrative tasks.

Principle 7: You are open and collaborative

There are many of businesses across all business sectors who are interested in using generative AI tools in their work. Your approach to any generative AI project should make use of existing cross-industry communities, where there is a space to solve problems collaboratively.

You should identify which groups and communities and other stakeholders that may have an interest in your project. You should have a clear plan for engaging and communicating with these stakeholders at the start of your work. 

You should seek to join cross-industry communities and engage with other organisations. Find other organisations who are trying to address similar issues and learn from them, and also share your insights with others. You should reuse ideas, code and infrastructure where possible.

Any automated response visible to the public such as via a chatbot interface or email should be clearly identified as such (e.g. “This response has been written by an automated AI-chatbot”)

You should be open with the public about where and how algorithms and AI systems are being used. 

Principle 8: You work with the right stakeholders from the start

Generative AI tools are new and you will need specific advice from commercial colleagues on the implications for your project. You should reach out to all stakeholders early in your journey to understand how to use generative AI in line with requirements. 

You should work with the right stakeholders to ensure that the expectations around the responsible and ethical use of generative AI are the same between developed AI systems and those purchased from a third party.

Principle 9: You have the skills and expertise needed to build and use generative AI

You should understand the technical requirements for using generative AI tools, and have them in place within your team. 

You should know that generative AI requires an understanding of new skills such as prompt engineering and you, or your team, should have the necessary skill set. 

You should take part in available training courses on generative AI, and proactively keep track of developments in the field. 

Principle 10: You use these principles alongside your organisation’s policies and have the right assurance in place

These principles and this framework set out a consistent approach for the use of generative AI tools for business. While you should make sure that you use these principles when working with generative AI, many organisations have their own governance structures and policies in place, and you also should follow any organisation-specific policies. 

You need to understand, monitor and mitigate the risks that using a generative AI tool can bring. You need to connect with the right stakeholders early in the project lifecycle for your generative AI tool.

You need to have clearly documented review and escalation processes in place, this might be a AI review board, or a project-level steering team.

Understanding generative AI

This section explains what generative AI is, the applications of generative AI in government and the limitations of generative AI and LLMs. 

It supports: Principle 1: You know what generative AI is and what its limitations are.

This section is centred on explaining generative AI and its limitations. You can find explanations of the core concepts around managing, choosing and developing generative AI solutions in the Building generative AI Solutions section.

What is generative AI?

Generative AI is a form of Artificial Intelligence (AI) - a broad field which aims to use computers to emulate the products of human intelligence - or to build capabilities which go beyond human intelligence.

Unlike previous forms of AI, generative AI produces new content, such as images, text or music. It is this capability, particularly the ability to generate language, which has captured the public imagination, and creates potential applications across all industry sectors.

Generative AI fits within the broader field of AI as shown below:

Models which generate content are not new, and have been a subject of research for the last decade. However, the launch of ChatGPT in November 2022 increased public awareness and interest in the technology, as well as triggering an acceleration in the market for usable generative AI products. Other well known generative AI applications include Claude, Bard, Bedrock, and Dall-E. These applications are a type of generative AI known as a Large Language Model (LLM).

Public LLM interfaces fit within the field of generative AI as shown below: 

Foundation models are large neural networks trained on extremely large datasets to produce responses which resemble those datasets. Foundation models may not necessarily be language-based, and they could have been trained on non-text data, e.g. biochemical information.

Large Language Models (LLMs) are foundation models specifically trained on text and natural language data to generate high-quality text based outputs.

Nested Language Models (NLMs) are trained to organise multiple models for natural language processing.

Applications Programme Interfaces (APIs) allows applications to talk to language models through secure connections.

User interfaces for foundation models & LLMs, are user-friendly ways that people without technical experience can use foundation models or LLMs. ChatGPT and Bard (now Gemini) are examples of these, at present they are mostly accessed by tool-specific URLs, but they are likely to be embedded into other consumer software and tools in the near future.

Generative AI works by using large quantities of data, often harvested from the internet, to train a model in the underlying patterns and structure of that data. After many rounds of training, sometimes involving machines only, sometimes involving humans, the model is capable of generating new content, similar to the training examples. 

When a user provides a prompt or input, the AI evaluates the likelihood of various possible responses based on what it has learned from its training data. It then selects and presents the response that has the highest probability of being the right fit for the given prompt. In essence, it uses its training to choose the most appropriate response for the user’s input.

Applications of generative AI in business

Despite their limitations, the ability of LLMs to process and produce language is highly relevant to the work of government, and could be used to:

However, LLMs and other forms of generative AI still have limitations: you should make sure that you understand these, and that you build appropriate testing and controls into any generative AI solutions.

Limitations of generative AI and LLMs

LLMs predict the next word in a sequence: they don’t understand the content or meaning of the words beyond how likely they are to be used in response to a particular question. This means that even though LLMs can produce plausible responses to requests, there are limitations on what they can reliably do.

You need to be aware of these limitations, and have checks and assurance in place when using generative AI in your organisation.

These limitations mean that there are types of use cases where you should currently avoid using generative AI, such as safety-of-life systems or those involving fully automated decision-making which affects individuals. 

However, the capabilities and limitations of generative AI solutions are rapidly changing, and solution providers are continuously striving to overcome these limitations. This means that you should make sure that you understand the features of the products and services you are using and how they are expected to change.

Building generative AI solutions

This section outlines the practical steps you’ll need to take in building generative AI solutions, including defining the goal, building the team, creating the generative AI support structure, buying generative AI and building the solution.

It supports:

However, following the guidance in this section is only part of what is needed to build generative AI solutions: you also need to make sure that you are using generative AI safely and responsibly.

Defining the goal

Like all technology, using generative AI is a means to an end, not an objective in itself. Whether planning your first use of generative AI or a broader transformation programme, you should be clear on the goals you want to achieve and particularly, where you could use generative AI.

Goals for the use of generative AI may include improved public services, improved productivity, increased staff satisfaction, increased quality, cost savings and risk reduction. You should make sure you know which goal you are seeking, and how you will measure outcomes.

Identifying use cases

When thinking about how you could leverage generative AI in your organisation you need to consider the possible situations or use cases. The identification of potential use cases should be led by business needs and user needs, rather than directed by what the technology can do. Encourage business units and users to articulate their current challenges and opportunities. Take the time to thoroughly understand users and their needs as per the Service Manual to make sure you are solving the right problems. Try to focus on use cases that can only be solved by generative AI or where generative AI offers significant advantages above existing techniques.

The use of generative AI is still evolving, but the most promising use cases are likely to be those which aim to:

Use cases to avoid

Given the current limitations of generative AI, there are many use cases where its use is not yet appropriate, and which should be avoided:

This list is not exhaustive: you should make sure that you understand the limitations of generative AI, as well as the features and roadmap of the products and services you are using.

Practical recommendations

Building the team

While public-facing generative AI services such as ChatGPT are easy to use and access, building production-grade solutions which underpin services to citizens requires a range of skills and expertise.

You should aim to build a multi-disciplinary team which includes:

You should ensure that you not only have the team in place to build your generative AI solution, but that you have the capability to operate your solution in production.

As well as building a team which contains the right skills, you should strive to ensure that your team includes a diversity of groups and viewpoints, to help you stay alert to risks of bias and discrimination.

Generative AI is a new technology, and even if you have highly experienced experts in your team, they will likely need to acquire new skills.

Acquiring skills

To help you acquire the specific skills needed to build and run generative AI solutions, you should consider implementing training resources:

You also consider a series of on-demand courses on more specific aspects of generative AI. 

You should tailor your learning plan to meet the needs of five groups of learners:

Creating the generative AI support structure

As generative AI is a new technology, you should make sure that you have the structures in place to support its adoption. These structures do not need to be fully mature before your first project: indeed, your experience in your first project will shape the way you organise these structures. However, you should ensure that you have sufficient control to make your use of generative AI safe and responsible.

The supporting structures required for effective generative AI adoption are the same as those required to support the broader adoption of other forms of AI. If your organisation is already using other forms of AI, these structures may already be in place.

If you do not already have them in place, you should consider establishing:

Buying generative AI

The generative AI market is still new and developing engagement with legal and procurement teams is particularly important to discuss partners, vendors, pricing, products and services.

They will help you navigate procurement in an emerging market and regulatory and policy landscape, as well as ensure that your procurement is aligned with ethical principles.

Specifying Your requirements

When buying AI products and services, you will need to document your requirements to tell your suppliers what you need. 

When drafting requirements for generative AI, you should:

Running your procurement

Having prepared your procurement strategy, defined your requirements, and selected your commercial agreement, you can now proceed to conduct a ‘call-off’ in accordance with the process set out in the relevant commercial agreement. The commercial agreement will specify whether you can ‘call-off’ by further competition, a direct award or either. 

Building the solution

Core concepts

Generative AI provides a wide breadth of capability, and a key part of designing and building a generative AI solution will be to get it to behave accurately and reliably. This section sets out key concepts that you need to understand to design and build generative AI solutions that meet your needs. 


Generative AI can be accessed and deployed in many different ways or patterns. Each pattern provides different benefits and presents a different set of security challenges, affecting the level of risk that you must manage. 

This section explains patterns and approaches as the main ways that you are likely to use and encounter generative AI, including:

Public generative AI applications and web services

Applications like OpenAI’s ChatGPT, Google’s Gemini, Microsoft’s Bing search, Claud are the consumer side of generative AI. They have a simple interface, where the user types in a text prompt and is presented with a response. This is the simplest approach, with the benefit that users are already familiar with these tools. 

Many LLM providers offer web services free of charge, allowing users to experiment and interact with their models. Generally, you’ll just need an email address to sign up.

There are a few things you’ll need to consider before signing up to a generative AI web service:

Embedded generative AI applications

LLMs can be embedded, or integrated, into existing and popular products; embedded generative AI allows people to use language-based prompts to ask questions about their organisation’s data, or for specific support on a task.

Embedded generative AI tools provide straightforward user interfaces in products that people are already familiar with. They can be a very simple way to bring Generate AI into your organisation. Examples of embedded generative AI tools include:

You must be certain you understand the scope of access and data processing of these services. Most enterprise licenced services will assure your control over your data. However, supporting services like abuse monitoring may still retain information for processing by the vendors.

If data sovereignty is a concern, you must also clarify the data processing geolocation with a vendor.

LLMs that are integrated into organisations existing enterprise licences may have access to the data that’s held by your organisation by default. Before enabling a service, you must understand what data an embedded generative AI tool has access to in your organisation.

The use of code assistance tools requires the addition of integrated development environment (IDE) or editor plugins. You must be certain to only use official plugins. If you use a coding assistant to generate a complex algorithm, it may be necessary to verify the licensing status manually by searching for the code on the internet to double-check you’re not inadvertently violating any copyrights or licences.

Public generative AI APIs

Most big generative AI applications will offer an Application Programming Interface (API). This allows developers to integrate generative AI capabilities directly into solutions they build. It takes only a few lines of code to build a plugin to extend the features of another application.

As with web services, signing up is typically required to obtain an access token. You need to be aware of the terms and conditions of using the API.

By using an API your organisation’s data is still sent over to the provider, and you must be sure that you are comfortable with what happens to it before using an API. 

The benefit of using APIs is that you will have greater control over the data. You can intercept the data being sent to the model and also process the responses before returning them to the user. This allows you to e.g.:

However, you will also need to perform additional tasks commonly performed by the user interface of web and embedded services, such as:

Local development

For rapid prototyping and minimum viable product (MVP) studies, the development on personal or local hardware (i.e. sufficiently powerful laptops) may be a feasible option.

Development best practices like distributed version control systems, automated deployment, and regular backups of development environments are particularly important when working with personal machines.

When working on local development you should consider containerisation and  cloud-native technology paradigms like twelve-factor applications. These will help when moving solutions from local hardware into the cloud.

Please note that the recommendation for production systems remains firmly with fully supported Cloud environments.

Cloud solutions

Cloud services provide similar functionality to public and paid-for APIs, often with a familiar web interface with useful tools for experimentation. Their key advantages is that they allow increased control over your data. You can access cloud service providers’ LLMs by signing up through your organisation’s AWS, Microsoft or Google enterprise account.

When establishing your generative AI cloud service, make sure the development environment is compliant with your organisations’ data security policies. 

If your organisation and/or use case requires all data to remain on UK soil, you might need to plan in additional time for applying for access to resources within the UK as these may be subject to additional regulation by some providers. Technical account managers and solution architects supporting your enterprise account will be able to help with this step.

Picking your tools

In order to develop and deploy generative AI systems you will need to pick the right tools and technology for your organisation. Deciding on the best tools will depend on your current IT infrastructure, level of expertise, risk-appetite and the specific use cases you are supporting.

Decisions on your development stack.

There are a number of technology choices you will need to consider when building your generative AI solutions, including the most appropriate IT infrastructure, which  programming languages to use and the best large language model.

Items for consideration include:

The choice of a generative AI framework might depend on:

Things to consider when evaluating Large Language Models (LLMs)

There are many models currently available so you need to select the most appropriate for your particular use case.  The Stanford Center for Research on Foundation Models (CRFM) provides the Holistic Evaluation of Language Models (HELM) to benchmark different models against criteria such as accuracy, robustness, fairness, bias, and toxicity; and can help you to compare the capabilities of a large number of language models. Here are some of the things you should consider: 

Getting reliable results

Generative AI technology needs to be carefully controlled and managed in order to ensure the models behave and perform in the way you want them to, reliably and consistently. 

There are a number of things you can do to help deliver high quality and reliable performance.

For further information see the deep dive section.

Testing generative AI solutions

Generative AI tools are not guaranteed to be accurate as they are designed to produce plausible and coherent results. They generate responses that have a high likelihood of being plausible based on the data that they have processed. This means that they can, and do, make errors. In addition to employing techniques to get reliable results, you should have a process in place to test them.

During the initial experimental discovery phases you should look to assess and improve the existing system until it meets the required performance, reliability and robustness criteria. 

Technical methods and metrics for assessing bias in generative AI are still being developed and evaluated. However, there are existing tools that can support AI fairness testing, such as IBM fairness 360, Microsoft FairLearn, Google What-If-Tool, University of Chicago Aequitas tool, and PyMetrics audit-ai. You should carefully select methods based on the use case, and consider using a combination of techniques to mitigate bias across the AI lifecycle.

Data management

Good data management is crucial in supporting the successful implementation of generative AI solutions. The types of data you will need to manage include:

Data management needs to also address data loss prevention. Consider using privacy enhancing technology to prevent data leakage, and if you process personal identifiable information take action to protect peoples’ data e.g. pseudonymising data to reduce the risk of leaking sensitive information.

Using generative AI safely and responsibly

This section outlines the steps you’ll need to ensure that you Build generative AI solutions in a safe and responsible way, taking account of Legal considerations, Ethics, Data protection and privacy, Security and Governance. Many of these considerations interact with each other, so you should read all of these topics together, and seek support from data ethics, privacy, legal and security experts.

It supports:

Legal considerations

You should seek advice from legal advisers who can help you to navigate through the use of generative AI. 

Although generative AI is new, many of the legal issues that surround it are not. For example, many of the ethical principles discussed in this document, such as fairness, discrimination, transparency and bias, have sound foundations in public law. In that way, many of the ethical issues that your team identifies will also be legal issues, and your lawyers will be able to help to guide you through them. 

The lawfulness and purpose limitation section provides a framework to ensure that personal data is processed lawfully, securely and fairly at all times. Your lawyers can advise you on that. 

You may face procurement and commercial issues when buying generative AI  products. Alongside commercial colleagues, your lawyers can help you to navigate those challenges.

When you contact your legal team, you should explain your aims for the generative AI solution, what it will be capable of doing, and any potential risks you are aware of. This will help you to understand, for example, if you need legislation to achieve what you want to do. It will also help to minimise the risk of your work being challenged in court, having unintended - and unethical - consequences or a negative impact on the people you want it to benefit.

Example legal issues

These are example legal issues designed to help you understand when you might want to consider getting legal advice. They should not be read as real legal advice and their application to any given scenario will be fact specific. You should always consult your departmental lawyer if in doubt.

Data protection

Data protection is a legal issue, with potentially serious legal consequences should the government get it wrong. Although your organisation will have a data protection officer and there may also be experts in your team, your legal team will be able to help you to to unpick some of the more difficult data protection issues that are thrown up by the use of generative AI.

See data protection and privacy section for more information.

Contractual issues

Your lawyers will help you to draw up the contracts and other agreements for the procurement or licensing of generative AI tools. There may be special considerations for those contracts, such as how to apportion intellectual property and how to ensure the level of transparency that would be required in a legal challenge. Contracts for technology services may need to incorporate procedures for system errors and outages, that recognise the potential consequences of performance failures.

See buying generative AI section for more information.

Intellectual property and copyright

The potential intellectual property issues with generative AI have been much discussed. Your lawyers can help you to navigate these, for example by considering at the outset how ownership of intellectual property rights and liabilities will be apportioned throughout the lifetime of the project. They can also give you advice on any copyright issues with the use of these systems in government.

Equalities issues

Lawyers can help you to navigate the equalities issues raised by the use of generative AI in government, for example obligations arising under the Equality Act 2010. Conducting an assessment of the equalities impacts of your use of generative AI can also be one way to guard against bias, which is particularly important in the context of generative AI.

If approached early, before contracts are signed, your legal advisers can help you ensure the government is fulfilling its responsibilities to the public to assess the impacts of the technology it is using.

Public law principles

Public law principles explain how public bodies should act rationally, fairly, lawfully and compatibly with human rights. These are guidelines for public bodies on how to act within the law. Many of these public law principles overlap with the ethical principles set out in this guidance. 

As a result, your lawyers will likely be able to guide you on the application of the ethical principles, based on their knowledge of public law and the court cases that have occurred and the detail of the judgments.

For example, public law involves a principle of procedural fairness. This is not so much about the decision that is eventually reached but about how a decision is arrived at. A correct procedure would ensure that relevant considerations are considered. The transparency and explainability of the AI tool may well be key in being able to demonstrate that the procedure was fair.

Public law also considers rationality. Rationality may be relevant in testing the choice of generative AI system; considering the features used in a system; and considering the outcomes of the system and the metrics used to test those outcomes. 

Where you are considering using generative AI in decision-making in particular, public law also can guide you for example on whether particular decisions require the exercise of a discretion by a decision maker, which could be unfairly fettered by the use of a tool, or whether in fact the decision can be delegated at all.

Human rights

Public authorities must act in a way that is compatible with human rights. It’s possible that AI systems (especially those involving the use of personal data) may in some way affect at least one of the European Convention on Human Rights (ECHR) of individuals. Examples of those most likely to be commonly impacted are Article 8 (right to a private and family life) and Article 10 (freedom of expression).


Sometimes, in order to do something, a public authority needs a legislative framework. Your lawyers will be able to advise you whether your use of generative AI is within the current legal framework or needs new legislation. For example, it may be that the legislative framework does not allow the process you are automating to be delegated to a machine. Or, it may be that it provides for a decision to be made by a particular person.


The ethical questions raised by your use of generative AI will depend on your context and the nature of your solutions. The key themes you should address include:

As well as the guidance in this framework, you should also take existing guidance into account the five cross-sectoral, values-based principles for responsible AI innovation set out in the AI regulation white paper. The paper provides a useful explainer for safety, security and robustness; appropriate transparency and explainability; fairness; accountability and governance; and contestability and redress.

Transparency and explainability

Transparency is a cornerstone of the ethical development, deployment and use of AI systems. A lack of transparency can lead to harmful outcomes, public distrust, a lack of accountability and ability to appeal. The AI regulation white paper establishes that AI systems should be appropriately transparent and explainable. Transparency is the  communication of appropriate information about an AI system to the right people. For example: information on how, when, and for which purposes an AI system is being used. Explainability is how much it is possible for the relevant people to access, interpret and understand the decision-making processes of an AI system.

However, transparency can be challenging in the context of generative AI, due to the closed and proprietary nature of commercial tools, and the inherent opacity of neural networks. You should therefore ensure that you are transparent about the design of the generative AI system and the processes in which it is embedded:

What you are transparent about:

How and to whom you are being transparent:

Although there are no universally accepted standards for achieving transparency in the use of generative AI, there are existing standards and external resources which you can draw on:

Accountability and responsibility

Ensuring accountability for generative AI means that individuals and organisations can be held accountable for the AI systems they develop, deploy, or use, and that human oversight is maintained. To establish accountable practices across the AI lifecycle, you should consider three key elements:

As an end-user, being accountable means taking responsibility for a system’s outputs and generated content and its potential consequences. This includes checking that these are factual, truthful, non-discriminatory, non-harmful, and do not violate existing legal provisions, guidelines, policies or the providers’ terms of use. It entails putting the necessary oversight and Human-In-The-Loop processes in place to validate output in situations with high impact or risk. Where these risks are too high, you must consider if generative AI should be used.

Ultimately, responsibility for any output or decision made or supported by an AI system always rests with the public organisation. Where generative AI is bought commercially, ensure that vendors understand their responsibilities and liabilities, put the required risk mitigations in place and share all relevant information. Refer to the buying generative AI section for further guidance.

Practical recommendations

Fairness, bias and discrimination

Fairness is a concept embedded across many areas of law and regulation, including equality and human rights, data protection, consumer and competition law, public and common law, and rules protecting vulnerable people. The AI regulation white paper sets out that AI systems should not undermine the legal rights of individuals or organisations, discriminate unfairly against individuals or create unfair market outcomes. 

Fairness, in the context of generative AI, means ensuring that outputs are unprejudiced, and do not amplify existing social, demographic, or cultural disparities.

By identifying and mitigating bias and reducing harm you will help your generative AI systems produce fairer outcomes. In generative AI, harmful biases can present as text, images, audio and video which perpetuate stereotypical or unfair treatment related to race, sex and gender, ethnicity, or other protected characteristics. Examples of this are the generation of harmful stereotypes or abusive content targeted against particular social groups.

Generative AI systems are designed, developed, and deployed by human beings who are bound by the limitations of their contexts and biases. They are always trained on data which encodes present and past biases and inequalities of society. These can present across the generative AI lifecycle, from data collection to prompt writing. The opacity and complexity of these systems can make it difficult to identify exactly where and how biases are introduced.

Generative AI models may reproduce biases embedded in training data or model design choices. They are particularly vulnerable to bias due to the fact that they are trained on vast amounts of unfiltered data scraped from the internet, which are likely to contain a wide range of content reflecting historical and social biases. The wording of prompts may also inadvertently introduce bias.

Addressing these issues can help to support equitable representation in AI-generated content. This might involve crafting prompts which encourage the consideration of different perspectives. For development teams, this might include ensuring training data is diverse, and implementing fairness testing to assess how the tool responds to different input. Technical methods and metrics for assessing bias in generative AI are still being developed and evaluated. Refer to the testing section for further guidance.

Practical recommendations

Information quality and misinformation

Having access to high quality information is vital to support effective decision-making. Generative AI poses a challenge to information quality due to its ability to generate content that appears credible but may be false or misleading.

The use of AI-generated content without proper validation and fact-checking can lead to the spread of misinformation. Many generative AI tools are built using large amounts of web-scraped data from unknown, potentially outdated and harmful, sources. For developers, this makes validating the data quality of generative AI models extremely difficult. 

The effectiveness of LLMs and other generative models is dependent on the quality of their training data. Even in cases where input data quality is deemed to be high, it is important to keep in mind that these tools cannot understand real-world contexts, nuances in language, cultural references, or intent and do not have access to information that is known to be real or true. LLMs are designed to generate statistically likely language patterns rather than producing reliable and truthful accounts of reality. This can make them convincing generators of ‘nonsense’. The tendency for generative AI models to present nonsensical or incorrect outputs as factual is sometimes referred to as ‘hallucination’.

To mitigate the risk of misinformation, you should check generated content for accuracy and truthfulness, and any potentially harmful or misleading information.

Practical recommendations

Maintaining appropriate human involvement in automated processes

Keeping a human-in-the-loop means ensuring that there is human involvement and supervision in the operations and outcomes of generative AI systems. In a broader context, humans should be involved with setting up the systems, tuning and testing the model so the decision-making improves, and then actioning the decisions it suggests. 

The availability of generative AI tools may contribute towards increasingly automated workflows and decision-making processes. However, relying on AI to make decisions and generate content without meaningful human oversight can have negative consequences. A lack of human intervention might result in inaccurate or harmful outputs going unchecked. You should assess the quality of AI-generated outputs to ensure they are accurate, relevant, and align with societal values. 

Generative AI also lacks flexibility, human understanding and compassion. While humans are able to take individual circumstances into account on a discretionary basis, AI systems do not have this capacity. 

Maintaining meaningful human involvement in generative AI ensures that future innovation aligns with human values and supports the public good. You should uphold the expectation ‘to be heard’ by a human when interacting and receiving services from the government. This supports the principle of transparency and building public trust. You should never use generative AI to fully automate decision making in high-risk or high-impact situations.

Practical recommendations

Sustainability and environmental considerations

Generative AI has environmental impacts that you and your organisation should understand and consider before deciding to develop or use generative AI solutions. Large language models, in particular, rely heavily on computational power both during their training phase and then every time they are used, contributing to carbon emissions. They may require the use of a lot of water to cool the data centres, and the manufacturing process of key components like the graphics processing units (GPUs) also contributes to the extraction of rare metals. 

You should balance the environmental costs of using pre-trained models and usage costs when deciding on the most appropriate model size for your needs. In general it will not be an environmentally-sound decision to train your own model if appropriate pre-trained models are available. As models are generally expensive to operate, they should not be used for tasks that could be undertaken by other available machine learning tools.

Generative AI can potentially contribute to reducing environmental impact as well. It can optimise processes and minimise resource wastage. For example, AI technologies can streamline data analysis, reducing the computational power required to process information. This optimisation results in lower energy consumption and a decreased carbon footprint.

Practical recommendations

Data protection and privacy

Generative AI systems can process personal data during their training and testing phases; as well as potentially generating outputs which contain personal data, including sensitive personal data. When using generative AI you need to consider how you protect personal data, are compliant with data protection legislation and minimise the risk of privacy intrusion from the outset. 

Organisations developing and deploying generative AI systems must consider principles of data protection outlined in the UK General Data Protection Regulation (UK GDPR) and the Data Protection Act 2018.

The data protection law applies irrespective of the type of technology used, so its basic principles of compliance will also apply to any generative AI systems. The Information Commissioner’s Office, which is responsible for regulating compliance with the data protection legislation in the UK, outlines these principles in their guidance.

The data protection principles most relevant to the use of generative AI are:


Accountability is a key principle in data protection law and the AI regulation white paper.  Accountability establishes ownership of risk, responsibility for mitigations and compliance with the legislation, ability to demonstrate your compliance and high standards for privacy. The AI regulation white paper notes that clear lines of accountability need to be established across the AI life cycle.

Organisations should take the following steps when planning generative AI solutions:

Practical recommendations

Lawfulness and purpose limitation

The nature of generative AI means that its misuse may result in high risks to data subjects. As a result, a Data Protection Impact Assessment (DPIA) should be undertaken prior to deploying any generative AI capabilities which process personal data. 

The DPIA process should identify personal data processing at each stage of the generative AI lifecycle starting from design to data acquisition and preparation, training, testing, deployment and monitoring. 

If you are processing personal data in your generative AI system that is not fully anonymised, you must identify an appropriate lawful basis under UK GDPR.  

The UK GDPR requires data controllers:

Identification of all personal data sources is important as data controllers will be accountable for all personal data processed throughout the generative AI lifecycle.  For example, generative AI products are often trained on publicly available information drawn from the internet. Publicly available content which contains personal data may have been published in the public domain lawfully, but it is not currently agreed that the re-use of public personal data to train an LLM is lawful. Before re-using personal data in an LLM or generative AI system, you should seek data protection and legal expertise to consider and advise whether the re-use of that data is compatible with the purposes for which it was collected. 

Special category data is personal data that needs more protection because it is sensitive, such as health data. If your generative AI system needs to process special category data, you must be able to demonstrate that you meet one of the specific conditions in Article 9 of the UK GDPR. 

When mapping personal data flows, it is important to identify the geographic location of each distinct processing activity since the processing of data outside the United Kingdom will increase the risk of losing the protection of the UK data protection laws. Data controllers may need to bring in additional safeguards, such as International Transfer Data Agreements if personal data is being processed in jurisdictions where the data protection regime is not deemed to be adequate and transfers of personal data is restricted under Article 46 of the UK GDPR.

If having undertaken a Data Protection Impact Assessment, data protection risks remain “high” even after mitigations, and you cannot do anything to reduce it, prior consultation with the ICO is required under UK GDPR before processing of personal data can begin.

Practical recommendations

Transparency and individual rights

In addition to the ethical reasons for seeking transparency, organisations need to be transparent about how they process personal data in a generative AI system so that individuals can effectively exercise the rights granted to them by the UK GDPR.

This obligation applies to the direct collection of data from individuals and to personal data collected from other sources. The rights relating to personal data granted to individuals under data protection law apply wherever personal data is used at any of the various points in training, testing and deployment of an AI system.

The UK GDPR requires data controllers:

The data transformation processes involved in training a model may convert personal data into a less detailed form, making training data harder to link to a particular named individual. However, even without direct identifiers, individual level data that is rich in other variables may lead to inadvertent identification of people and is subject to data protection safeguards. This data needs to be considered when responding to individuals’ requests to exercise their rights as the initial processing stages may have included their personal data.

Practical recommendations


In addition to ethical reasons for fairness, it is also a data protection obligation under the UK GDPR for generative AI systems that process personal data. In the context of the data protection legislation, fairness means that “you should only process personal data in ways that people would reasonably expect and not use it in any way that could have unjustified adverse effects on them.”

You must make sure that generative AI systems do not process personal data in ways that are unduly detrimental, unexpected or misleading to the individuals concerned. You need to uphold the ‘right to be informed’ for individuals whose personal data is used at any stage of the development and deployment of generative AI systems as part of fulfilling the transparency and fairness principles.  

If generative AI systems infer data about people, you need to ensure that the system is accurate and avoids discrimination. Data protection aims to protect individuals’ rights and freedoms with regard to the processing of their personal data, not just their information rights. This includes the right to privacy but also the right to non-discrimination. 

Data Protection Impact Assessments (DPIAs) are the main tool to steer you to consider the risks to the rights and freedoms of individuals, including the potential for any significant social or economic disadvantage. DPIAs also help you to demonstrate whether your processing is necessary to achieve your purpose, is proportionate and fair. 

You must remember that there may be other sector-specific obligations around lawfulness, fairness, statistical accuracy or discrimination to consider alongside data protection obligations (e.g. Equality Act 2010). These are discussed in more detail under the Legal considerations section.

Practical recommendations

Data minimisation

The data minimisation principle requires you to identify the minimum amount of personal data you need to fulfil your purpose, and to only process that information, and no more. This does not mean that generative AI shouldn’t process personal data. If you can achieve the same outcome by processing less personal data then by definition, the data minimisation principle requires you to do so.

There are a number of techniques that you can adopt to develop generative AI systems that process only the data you need, while still remaining functional. The Centre for Data Innovation and Ethics (CDEI)’s responsible data access programme includes important work to encourage adoption of Privacy-Enhancing Technologies (PETs). PETs are a set of emerging techniques that provide stronger protections to preserve data privacy whilst enabling effective use of data. PETs come with their own limitations however, therefore selection of the PET technology should be proportionate to the sensitivity of the data.

CDEI has published a PET adoption guide to raise awareness of these emerging technologies. Similarly, the ICO has published the new PET guidance which explains how they can be used to support a data protection by design approach in line with regulatory requirements.

Practical recommendations

Storage limitation

Generative AI systems can only process personal data as long as you can reasonably justify it for the purpose you are processing. As challenging as it may be, you need to strike a delicate balance between any relevant training of large language models and minimising the collection and storage of personal data to meet the UK GDPR requirement of storage limitation.

It may be necessary to retain training data in order to retrain the model, for example when new modelling approaches become available and for debugging. However, where a model is established and unlikely to be retrained or modified, the training data may no longer be needed. You should: 

There are a number of strategies you can follow to address concerns around long (or even perpetual) retention of personal data. Storage limitation is best complied with through purpose limitation and data minimisation. You should map all personal data flows through stages of development, testing and deployment, and utilise data minimisation or eventually anonymisation techniques to remove or irreversibly transform personal data from training datasets.

Practical recommendations

Human oversight

Although it is possible to use generative AI systems for automated decision making where the system makes a decision automatically without any human involvement,  this may infringe the UK GDPR. Under Article 22, the UK GDPR currently prohibits “decision(s) based solely on automated processing” that have legal or “similarly significant” consequences for individuals. Services that affect a person’s legal status or their legal rights utilising generative AI must only use it for decision-support, where the system only supports a human decision-maker in their deliberation. 

Generative AI systems need to bring processes into training, testing and output stages so that humans work together with machines to perform tasks, combining their abilities to reach best results. However, the human input needs to be “meaningful”. The degree and quality of human review and intervention before a final decision is made about an individual are key factors in determining whether a generative AI system is being used for automated decision-making or merely as decision-support.

There are a number of factors that should determine the amount of human involvement in generative AI, such as the complexity of the output, its potential impact, the amount of specialist human knowledge required. As an example, generative AI systems deployed in legal, health and care are likely to always require human involvement no matter how exceptional the technology. 

While focusing on generative AI risks, it is important to consider biases at organisational and human review levels. Humans and generative AI technology have different strengths and weaknesses when it comes to ensuring fair outcomes. Generative AI cannot use emotional intelligence, nuance, or an understanding of the broader context. At the same time, humans have their own unconscious biases and beliefs that influence their reasoning. This points back to the importance of the accountability principle, robust governance structures for oversight and alignment of generative AI and existing business processes, such as risk management. 

Further aspects on human oversight for generative AI systems can be found in the ethics section.

Practical recommendations


Accuracy in the context of data protection requires that personal data is not factually  incorrect or misleading, and where necessary, is corrected, deleted and kept up to date without delay.

You need to put in place appropriate mathematical and statistical procedures as part of your technical measures to correct inaccuracies in personal data and minimise errors. Generative AI outputs should be tested against existing knowledge and expertise in early implementations of those outputs.

The outputs of a generative AI system are not always intended to be treated as factual information about the individual but instead represent a ‘statistically informed guess’. You need to factor in the possibility of them being incorrect and the impact this may have on any decisions. To avoid such misinterpretations of outputs as factual, systems should  be explicit that they are statistically informed guesses rather than facts, including information about the source of the data and how the inference has been generated.

For more information see the getting reliable results section.

Practical recommendations


Your organisation has a responsibility to ensure that the services it provides do not expose the public to undue risk, which makes security a primary concern for anyone looking to deploy emerging technology, such as generative AI. 

This section takes you through how to keep generative AI solutions in government secure:

How to deploy generative AI securely

Generative AI can be deployed in many different ways. The approaches set out below present different security challenges and can affect the level of risk that must be managed.

This section covers different approaches that you need to take for:

For additional information see the section on deployment patterns.

Public generative AI applications and web services

The use of public chatbots such as Google Gemini or ChatGPT are easier to use compared to open-source, bespoke solutions.

However, a key disadvantage of allowing the use of public applications is that you cannot easily control the data input to the models, and must rely on training users on what they can and cannot enter into the chat prompt. You also have no control on the outputs from the model and are subject to their commercial licence agreements and privacy statements, for example OpenAI will use the prompt data you enter directly into the ChatGPT website to improve their models, although individual users can opt out.

Embedded generative AI applications

As well as these more direct approaches to using generative AI, many vendors include generative AI features and capabilities directly within their products, for example Slack GPT and Microsoft 365 Copilot. Whilst this guidance applies at a high level to each of these applications, they come with their own unique security concerns, you should speak to your security teams to discuss your requirements.

In addition to embedded applications there are also many generative AI tools that offer plugins or extensions to other software, for example, Visual Studio Code has a large ecosystem of community built extensions, many of which offer generative AI functionality. Extreme caution should be taken before installing any unverified extensions as these are likely to present a security risk. You should speak to your security team to discuss your requirements.

Before adopting any of these products it is important to understand the underlying architecture of the solution and what mitigations the vendor has put in place for the inherent risks associated with generative AI.

All of these different approaches come with trade-offs between security, privacy, usability and cost. Each of the security risks of generative AI models need to be taken in context with the way the model is deployed and used to inform the level of risk that an application poses.

Public generative AI APIs

Many public generative AI applications usually offer the ability to access their services through Application Programming Interfaces (APIs), which define the set of rules, protocols, and tools for building software applications. Through using the API it can be very easy to integrate generative AI capabilities into your own applications. The benefit here is that you can intercept the data being sent to the model and also process the responses before returning them to the user. 

You can also include Privacy Enhancing Technology (PET) to prevent data leakage, add content filters to sanitise the prompts and responses, and it also means you can log and audit all interactions with the model. Note that PETs come with their own limitations, therefore selection of the PET should be proportionate to the sensitivity of the data: see ICO’s privacy-enhancing technologies (PETs) and CDEI’s PET adoption guide for more information.

Use of the API still means that data is passed over to the provider, although the retention policies tend to be more flexible for API use, for example, OpenAI only retains prompt data sent to the API for 30 days.

Privately hosted open source generative AI models

Instead of using a public generative AI offering, the alternative is to host your own generative AI model. By taking one of the many publicly available open source models and running it in your own private cloud infrastructure, you ensure that data never leaves an environment that you own. 

The type of models that you can run in this way are not on the scale of the publicly available ones, but can still provide acceptable results. The advantage is that you have complete control over the model and the data it consumes. The disadvantage is that you are responsible for ensuring the model is secure and up to date. 

An alternative approach is to use one of the larger commercial models, but in a private managed instance, for example, the Microsoft Azure OpenAI service offers access to the OpenAI ChatGPT models but running in a private instance with zero-day retention policies.

Data provenance

In addition to where your generative AI model runs, how the model was trained is also important from a security perspective. All the publicly available models were trained using data from the public internet. This means that they include data that is personally identifiable, inaccurate, illegal and harmful, all of which could present a security risk.

It is possible to train a LLM using your own data, but the cost of doing this for larger and more capable models is prohibitive.  Along with the cost, the amount of private data required to produce acceptable performance of a large model is also beyond the capacity of most organisations.

Working with your organisational data

A key application of generative AI is working with your organisation’s private data, by enabling the model to access, understand and use the private data, insights and knowledge can be provided to users that is specific to their subject domain and will provide more reliable results.

Open-source vs closed-source models

Neither open-source or closed-source LLMs are inherently less secure than the other. A fully open-source model may expose not only the model code, but also the weights of its parameters and the data used to train the model. While this increases transparency, it also potentially presents a greater risk, as knowing the weights and the training data could allow an attacker to create attacks carefully tailored to the specific LLM.

One benefit of fully open-source models is that they allow you to inspect the source code and model architecture, enabling security experts to audit the code for vulnerabilities. Despite this, owing to their complexity, even an open source LLM is mostly opaque, meaning that the internals of the model are hard to analyse. Open-source models theoretically benefit from a community of developers, who can quickly identify and fix security issues, whereas closed-source model owners might be incentivised not to publicise security flaws in their models. However, it should be noted that several high-profile vulnerabilities in open source libraries have been present for many years before being identified.

Security risks

Significant work has already been done by the Open Worldwide Application Security Project (OWASP) to identify the unique risks posed by LLMs. From these we can draw out some of the most common vulnerabilities and put them in context of how they could apply to LLM applications in government. These risks focus on the use of LLMs but many of them will also apply to other types of generative AI models. 

We take each security risk and use a scenario describing an application of generative AI in a government context, to illustrate how that vulnerability might be exploited. The list of scenarios is not exhaustive, but should be used as a template for assessing the risks associated with a particular application of generative AI. 

Impacts are described for each scenario, and mitigations suggested. The likelihood and impact of each risk in a given scenario are scored, following the approach outlined in the OWASP risk rating methodology. In addition to the impact factors included in the OWASP approach, we add user harm and misinformation as a significant impact factor. 

Security threats include:

Prompt injection threats

Prompt injections can either be direct, meaning a user directly enters a prompt into the LLM to subvert its behaviour. Or they can be indirect, meaning the LLM gets input from an external source, and that source has been manipulated to include a prompt injection, for example from an email or an external file.


Because of the risks around security, bias and data, all AI programmes need strong governance processes. In the AI regulation white paper, the accountability and governance principle says that governance should ensure effective oversight of the supply and use of AI systems, with clear lines of accountability established across the AI life cycle. Whether they are already built into existing governance frameworks or a new governance framework, the processes should be focused on: 

As part of any governance framework, organisations should consider setting up a separate AI Governance board or have AI representation on a Governance Board and an Ethics Committee. An AI governance board and an ethics committee are components of responsible AI implementation within an organisation or department which play different and distinct roles and responsibilities.

AI governance board or AI representation on an existing board

Ethics committee

The primary focus of an ethics committee is to assess the ethical implications of various actions, projects, and decisions within the organisation. It evaluates projects, policies, and actions from an ethical standpoint, focusing on values such as fairness, transparency, and privacy.

It typically includes legal experts, representatives from relevant organisations, community members, and other stakeholders who provide a specialised perspective on ethical matters and may also include Civil Society Organisations (CSOs).

See the ethics section for related content.

Creating an AI/ML systems inventory

To support the work, organisations should consider setting up Artificial Intelligence and Machine Learning systems inventory to provide a comprehensive view of all deployed AI systems within an organisation. 

It helps management and stakeholders understand the scope and scale of AI usage across programmes and projects, providing better oversight and awareness of any AI used in making decisions, and potential risks such as data quality, model accuracy, bias, security vulnerabilities, and regulatory compliance. The inventory should be regularly kept up to date with the following details:


Programme governance in teams should consider