Sample privacy impact assessment: an AI chat agent prepares quote requests

Example Brokerage, a property and casualty insurer, is adding an artificial intelligence chat agent to its website. The agent answers visitors' questions about home and car insurance and collects the information a quote request needs: name, contact details, address, vehicle and past claims. Example Brokerage is undertaking the project for two purposes: to answer visitors' questions outside business hours, and to prepare quote requests for its brokers. The information is not intended for any other purpose.

The agent is supplied by a Québec company and relies on a language model from an American vendor to draft its answers. The agent does not set premiums and does not turn clients away. A broker takes over every request and makes every decision.

This assessment covers the personal information handled through the chat agent. That includes collection from visitors, hosting by the chat agent provider, processing by the language model vendor, access by Example Brokerage staff, and retention of the conversations. The project is expected to involve between 100 and 10,000 persons. The decisions brokers make after taking over a request are not made by the agent and are not the subject of this assessment.

Personal information is collected directly from clients who visit the Example Brokerage website and use the chat agent. Through the conversation, the agent gathers their name, email address, phone number and home address, along with details of their vehicle and their past claims. The content of the conversations is itself retained as personal information. The persons concerned are not currently informed of how this information is handled.

The conversations are hosted in the cloud by the chat agent provider, a company located in Québec, which keeps the conversations under a written agreement with Example Brokerage. To draft the agent's answers, the content of the conversations is sent to a language model vendor located in the United States. Whether a written agreement exists with that vendor remains to be confirmed. It also remains to be confirmed whether the vendor keeps the conversations and, if so, for how long.

Within Example Brokerage, access to the information is limited to the staff who need it. The information comes back to the organisation as quote requests prepared for the brokers, who take over each request and make every decision. Visitors receive the answers drafted by the agent. No retention period has been defined for the conversations and the information collected, and this remains to be confirmed. It also remains to be confirmed whether the information collected can be communicated to the persons concerned in a structured format.

The assessment brought four findings to light. First, it has not been confirmed that a written agreement exists with the language model vendor, which processes the conversations outside Québec. Second, no retention period has been defined. Third, the persons concerned are not informed. Fourth, it is not known whether the information can be communicated in a structured format.

The main risks identified are the processing of information outside Québec without adequate protection, which is of medium severity and medium likelihood, and retention longer than necessary, also of medium severity and medium likelihood. A risk of unauthorised access to the information was also identified, of medium severity and low likelihood. The overall sensitivity of the information is assessed as medium. Several points remain to be confirmed, notably whether the American vendor keeps the conversations and for how long.

Example Brokerage adopts the five measures set out in this document to address these findings and risks.