Privacy considerations when developing AI
Things to know
- Canadian privacy laws apply to the development of AI if personal information is processed, for example, if personal information is used to train an AI model, if an AI model or system processes personal information, or if an AI model or system generates output that includes personal information.
- “Personal information” is broadly defined under Canadian privacy laws and includes information that can be used alone or in combination with other information to identify an individual.
- Unlike under privacy laws in some jurisdictions, such as the European Union, Canadian privacy laws are consent-based with limited exceptions to consent and do not include a “legitimate interests” legal basis for processing personal information.
- Publicly available personal information is subject to consent and other obligations under privacy laws with limited exceptions.
- It is uncertain whether or to what extent “implied consent” may be relied upon as legal authority for collecting or using personal information to train an AI model or generate output, or whether consent to use personal information to train an AI model or generate output can be made a “condition of service” (with no ability to opt out).
Things to do
- Identify and document your legal authority (i.e., consent or an exception to consent) for collecting and using personal information to train an AI model or generate output. When relying on consent, ensure that the consent is valid and meaningful. When sourcing personal information from a third party, obtain assurances that the information was collected lawfully and the third party has authority to disclose it to you for the intended purposes (e.g., AI model training).
- Remember that, with limited exceptions, publicly available personal information, including information published online, is subject to privacy laws in Canada (including when subject to an exception to consent).
- Be open and transparent (e.g., in your publicly-posted privacy policy and user flows) about what, how, when and why personal information is collected, used or disclosed during development, training or operation of the AI model or system, and provide this information in an understandable manner. Disclose any known limitations about the accuracy of AI outputs (e.g., age of data used to train the model) and any known or likely risks.
- Collect, use and disclose personal information only for documented, legitimate and appropriate purposes.
- Do not use personal information to develop or deploy AI systems for purposes that violate “no-go zones” identified by Canadian privacy regulators, such as “profiling that may lead to unfair, unethical or discriminatory treatment, or creating outputs that threaten fundamental rights and freedoms.”
- Use an adversarial testing process to identify potential unintended inappropriate uses of your AI model or system and, if applicable, take steps (such as technical measures or mandatory acceptable use policies) to prevent inappropriate uses.
- Collect, use, retain and disclose personal information only to the extent needed to fulfill the explicitly specified, appropriate purpose, including by removing personal information from datasets used to train AI models where possible and appropriate. Ensure that personal information used to train an AI model is accurate as necessary for the purpose of the model.
- Do not use personal information when using anonymized or synthetic data will allow you to achieve your identified purpose.
- When personal information is required, remove direct identifiersfrom the dataset wherever possible such that only de-identified personal informationremains.
- Perform (and update over time) privacy impact assessments, algorithmic impact assessments and bias testingand assess resilience to inferencing or other attacks.
- Develop a comprehensive AI governance program and/or review and enhance your privacy governance program and security policies and practices to address AI-related issues, including by enabling individuals to request access to their personal information, ask questions, submit complaints and correct inaccurate personal information.
- Consider if, or to what extent, laws of other jurisdictions (such as the E.U.’s Artificial Intelligence Act or General Data Protection Regulation) apply to an AI model or system that you make available to users outside Canada.
Useful resources
- “Principles for responsible, trustworthy and privacy-protective generative AI technologies,” Joint publication of the Office of the Privacy Commissioner of Canada and provincial data privacy regulators, December 7, 2023
- “Concluding joint statement on data scraping and the protection of privacy,” Joint publication of the Office of the Privacy Commissioner of Canada and international data privacy regulators, October 2024
Next
