From Customer Data to AI Processing

Every AI powered startup wants to build better products. One of the fastest ways to improve an AI product is to learn from customer interactions and data. Yet, many founders assume that if they already comply with the GDPR or the CCPA as a Data Processor or Service Provider, they may also use that information to train their models or improve products without further privacy law analysis. That assumption is incorrect. AI has expanded the range of processing activities performed using personal information, making it no longer sufficient to analyze compliance solely at the point of collection. Instead, founders should analyze privacy compliance on a processing-activity basis, evaluating each new use of personal information rather than assuming compliance for one processing activity automatically extends to the other.

For years, compliance assessments were often centered on the personal information that a data controller or business collected from its users through contact forms, account registrations, IP addresses, browsing activity, or other internet interactions. Today, AI-enabled analytics, chatbots, and other AI tools use that same information often together with prompts, uploaded documents, and conversation histories to perform new processing activities, including generating inferences, profiling users, and predicting behavior. As a result, founders should no longer limit their privacy analysis to “collection activities” but to each AI-enabled “processing activity” performed using the information.

The GDPR and CCPA have always regulated data collection and processing, so the analytical framework generally remains the same. However, your company’s AI activities change the factual analysis and how you answer each question of the framework.

Consider a startup that provides AI-powered analytics to enterprise customers under a service provider agreement. Initially, the startup processes customer data solely to generate reports for that customer in accordance with and pursuant to a written contract. Months later, it decides to use the same customer data to train its own large language model and improve AI products for future customers. Although the information being processed has not changed, the purpose for which it is processed has. That new processing activity may require the startup to reassess whether it remains a Service Provider or Data Processor for that activity and whether additional obligations arise under the CCPA or the GDPR. The analysis also depends on whether the information continues to qualify as personal information under the applicable privacy law. If the startup instead uses anonymized or deidentified information, the legal analysis may differ significantly.

Thus, AI changes the factual analysis by expanding what startups can process and do with the information they collect, often creating new processing activities that carry independent privacy implications under the GDPR and CCPA. These new purposes may require startups to re-evaluate the legal role they occupy whether as a service provider, processor, business, or controller for each AI-enabled processing activity and the nature of the information being used (personal information, deidentified data, or aggregate consumer information). Each AI feature, model, training activity, or workflow may involve a distinct processing purpose or activity that should be evaluated under the GDPR and CCPA. Depending on how the AI is designed and used, those activities may introduce additional privacy obligations, such as increased transparency obligations, contractual requirements, consumer rights, purpose limitation considerations, or data governance responsibilities.

This Article develops a practical framework to help startup founders analyze AI systems under the GDPR and CCPA. Although startups should also evaluate other applicable federal, state, and international privacy laws depending on their operations and the jurisdictions in which they do business, the GDPR and the CCPA provide a useful framework for understanding how AI changes the privacy analysis.

This Article first explains how founders should determine whether their startups handle information regulated by the GDPR and CCPA. It next examines the legal roles startups may occupy under each statute because those roles determine the obligations that apply to each processing activity. Finally, it demonstrates how AI-enabled features including inference generation, profiling, AI model training, and automated data collection may alter the privacy analysis by introducing new processing activities and purposes that require startups to reassess their obligations under existing law.

Section I. Does my company collect or process personal information regulated by the GDPR and CCPA?

The first step in the privacy analysis is determining whether a start-up collects or processes information governed by the GDPR and the CCPA. Simply collecting or processing personal data or information, as defined by the statutes, however, does not automatically mean that a startup is subject to either regulation. Rather, a startup must both handle regulated (a) personal information and (b) fall within the jurisdictional scope of the GDPR or the CCPA.

(a) Personal Information

Both statutes define regulated information broadly. The GDPR protects “personal data¹” while the CCPA protects “personal information²” and each definition extends beyond obvious identifiers such as name, email addresses, physical addresses and precise location, to include any information that can directly or indirectly identify an individually. Both statutes also exclude deidentified or anonymous information such that it no longer identifies or can reasonably be used to identify an individual.

Accordingly, founders should first determine whether their startup handles regulated information. Then, they should determine whether the startup falls within the jurisdictional scope of the GDPR or the CCPA because each statute applies under different jurisdictional standards.

(b) Jurisdiction

GDPR Jurisdiction. “Does the startup process the personal data of individuals located in the European Union in connection with EU-directed activities?”

The GDPR may govern a U.S. start-up company for two primary reasons³.

First, the company has an establishment in the EU, such as an office, branch, or subsidiary, and processes personal data in connection with that establishment, regardless whether the actual processing takes place in the EU.

Second, even if the startup has no establishment in the European Union, the GDPR may apply when the startup, acting as a non-EU controller or processor, processes the personal data of individuals located in the European Union in connection with either (a) offering them goods or services, regardless of whether payment is required, or (b) monitoring their behavior within the European Union.

Applicability for Non-EU Businesses

This second scenario is most relevant to U.S. startups because many products and online services are available to users worldwide, and the GDPR will apply to all data subjects located in the EU, regardless of their citizenship, legal status, or residence. For example, GDPR applies to a US-based company targeting EU-located tourists with advertisements, even if the tourists are from outside the EU⁴.

Recital 23 of the EU GDPR makes it clear that a business must show intent to offer goods or services to data subjects in the EU.⁵ Accordingly, a U.S. startup is not subject to the GDPR merely because an individual located in the EU happens to visit its website and submit personal information. Instead, the relevant inquiry is whether the startup’s activities demonstrate an intent to target individuals in the EU as customers. The following factors help founders determine whether their startup has the requisite intent to offer goods or services to individuals in the EU⁶:

• Offering goods or services in an EU language or currency
• Allowing data subjects in the EU to place orders in the local language
• Referring to the EU or at least one EU member state by name when referencing the goods or services
• Offering delivery in EU member states
• Directing marketing campaigns at an EU member state
• Using country-specific top-level domains or the top-level domain “.eu”.

Applicability for Non-EU Processors

Additionally, under this second scenario, both the activities of the processor and the controller should be considered when determining if the GDPR applies to the data processing activities of the processor. For example, a US cloud provider will be subject to the GDPR when it provides data storage services to a U.S. health and lifestyle app developer that monitors app user behavior in the EU⁷. The European Data Protection Board (“EDPB”) interprets Article 3(2) to apply because the processor's processing activities are related to the app developer's (controller’s) offering of services to, and monitoring of, individuals in the European Union. Thus, the territorial scope analysis focuses on the relationship between the processor's processing activities and the controller's EU-directed activities, rather than requiring the processor itself to independently target individuals in the European Union.

CCPA Jurisdiction. “Does the business collect personal information of California residents and satisfy the obligations under the statutory role it occupies?”

Unlike the GDPR, the CCPA does not determine its applicability based on whether a business intentionally targets California consumers. Instead, the CCPA may apply when the business collects or processes personal information of California residents⁸ in connection with business activities in California and qualifies as a Business or otherwise acts as a Service Provider, Contractor, or Third Party under the statute.

Once a startup falls within the CCPA’s jurisdiction, it may be regulated in one of several statutory roles, either as:

1) A “Business”: a for-profit entity that (1) collects CA consumers’ personal information, (2) determines the purposes and means of the processing of consumers’ personal information, (3) does business in the state of CA, and (4) satisfies one or more of the following thresholds:

• As of January 1 of the calendar year, had annual gross revenues in excess of twenty-five million dollars ($25,000,000) in the preceding calendar year, as adjusted pursuant to subdivision (d) of Section 1798.199.95.
• Alone or in combination, annually buys, sells, or shares the personal information of 100,000 or more consumers or households.
• Derives 50 percent or more of its annual revenues from selling or sharing consumers' personal information; or

2) A (a) “Service Provider”, (b) “Contractor”, or (c) “Third party”.

Quick Comparison

Assuming the GDPR applies, there is no revenue or processing-volume threshold for qualifying as a Data Controller, unlike the CCPA's threshold for qualifying as a Business. Rather, under the GDPR, an entity is a controller if it determines the purposes and means of processing personal data, regardless of its size⁹. The CCPA’s tiered framework helps smaller U.S. businesses navigate privacy compliance by limiting the definition of a “Business” to entities that satisfy specified statutory thresholds while assigning different obligations based on an entity’s statutory role.

In short, the GDPR determines its applicability by examining whether a startup's processing activities are sufficiently connected to the European Union, either because they are carried out in the context of an EU establishment or because they are related to the intentional offering of goods or services to, or monitoring the behavior of, individuals in the European Union. By contrast, the CCPA determines its applicability by examining whether it processes the personal information of California residents and, if so, whether it falls within one of the statute’s regulated entity classifications.

Section II. Which Role Does My Startup Play?

Once founders determine that the GDPR or CCPA applies to their company, they must identify the company’s legal role with respect to the processing of personal information because that classification determines the obligations that follow. Under the GDPR, covered start-ups generally act as either controllers or processors.

Under the CCPA, covered start-ups generally function as Businesses, Service Providers, Contractors, or Third Parties. These classifications are defined differently and carry different statutory responsibilities.

GDPR Roles

The GDPR principally distinguishes between controllers and processors.

A Controller¹⁰ means the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data.

By contrast, a Processor¹¹ processes personal data on behalf of the controller. Although the GDPR also recognizes joint controllers and permits processors to engage sub-processors, most startup privacy analyses begin by determining whether the startup decides why and how personal data is processed (controller) or instead processes personal data on behalf of another entity (processor).

CCPA Roles

The CCPA employs a different framework by classifying entities as Businesses, Service Providers, Contractors or Third Parties.

A Business¹² is generally a for-profit legal entity that: (A) collects consumers' personal information (or has it collected on its behalf), (B) determines the purposes and means of processing that personal information, (C) does business in California, and (D) satisfies one or more of the CCPA's statutory applicability thresholds.

A Service Provider¹³ is a person or entity that: (A) processes personal information on behalf of a business, and (B) processes the information for a business purpose pursuant to a written contract that restricts the service provider's use, retention, and disclosure of the personal information.

Similarly, a Contractor¹⁴ means a person to whom a Business makes available a consumer's personal information for a business purpose pursuant to a written contract that satisfies the CCPA's statutory requirements. Like a Service Provider, a Contractor processes personal information on behalf of the Business and is contractually restricted in how it may retain, use, disclose, or otherwise process that information, but unlike a Service Provider, a contractor must certify that the contractor understands the restrictions in subparagraph (A) and will comply with them.

Finally, a Third Party¹⁵ means a person who is not any of the following:

1. The business with whom the consumer intentionally interacts and that collects personal information from the consumer as part of the consumer's current interaction with the business under this title.
2. A service provider to the business.
3. A contractor

Once a startup's role has been identified, the remainder of the privacy analysis becomes considerably more straightforward. Both the GDPR and the CCPA assign substantive compliance obligations according to an entity's legal role. Accordingly, founders should next determine which statutory obligations apply to the startup based on its classification as a controller, processor, Business, Service Provider, Contractor, or Third Party.

The specific obligations associated with each statutory role are beyond the scope of this article.

Section III. How does AI Change the Analysis?

Artificial intelligence does not alter the legal framework established by the GDPR or CCPA.

Startups must still answer the same threshold questions before determining their compliance obligations:

1. whether they are collecting or processing personal data or personal information regulated by the GDPR or the CCPA;
2. whether either statute applies to their processing activities; and
3. what legal role they occupy, such as controller, processor, Business, Service Provider, Contractor, or Third Party.

AI, however, fundamentally changes how these questions are answered.

Are we collecting personal information?

Historically, startups often evaluated privacy compliance by asking a relatively straightforward question: Are we collecting personal information? Registration forms, customer accounts, and checkout pages typically gathered obvious identifiers such as names, email addresses, telephone numbers, and mailing addresses. Modern AI systems, however, routinely process many additional categories of information including IP addresses, device identifiers, geolocation data, browsing behavior, prompts, uploaded documents, conversation histories, and other interaction data that may, individually or collectively, be reasonably linked to an identifiable individual.

AI expands the analysis from asking whether a startup collected a customer's name or email address to also asking what information its AI systems collect, process, and combine about individuals.

Consider a B2B software startup headquartered in Georgia. The startup has not yet signed any enterprise customers and does not require website visitors to create an account, complete a registration form, or purchase products through its website. It therefore assumes that neither the GDPR nor the CCPA applies because it does not collect names, email addresses, or other traditional customer information. The startup, however, actively markets its services to prospective business customers across the United States and in the European Union through an EU-directed website. The company's AI-enabled website analytics continuously analyze visitors' mouse movements, scrolling behavior, click patterns, and other interaction data generated simply by visiting the website combined with common device identifiers and IP addresses. The AI system predicts user intent, identifies the likely returning visitors, generates visitor profiles, personalizes website content, and recommends pricing offers. Although the startup never requested a visitor's name or email address, its AI has generated inferences about identifiable individuals. Whether the GDPR or the CCPA applies no longer turns solely on whether the startup collected names or email addresses. Instead, founders must evaluate whether the AI-generated interaction data, device identifiers, IP addresses, and resulting inferences constitute regulated personal information and whether the startup's activities bring it within the scope of the applicable statute.

The same analysis increasingly arises in AI-enabled customer support. Traditionally, a company's website might have contained only static frequently asked questions. Today, many startups deploy AI-powered chat assistants that answer customer questions, summarize conversations, retain prompts, generate recommendations, and maintain conversation histories. Even where the visitor never creates an account or affirmatively provides a name, the startup may nevertheless process IP addresses, persistent identifiers, and device information, that, taken together with AI prompt contents, conversation histories, and inferred interests, may qualify as personal information under the CCPA or personal data under the GDPR.

The CCPA’s broad definition of "personal information"¹⁶ expressly includes categories such as Internet or other electronic network activity, including internet protocol (IP) addresses¹⁷. Accordingly, the relevant privacy question is not simply whether a startup collected names or email addresses. Rather, startups must ask whether their AI systems are collecting, processing, combining, or inferring information that can reasonably be linked to identifiable individuals.

One might reasonably say that the collection of internet or website activity is not new. After all, companies operating websites and digital platforms have collected internet protocol (IP) addresses, cookies, device identifiers, and browsing activity for many years. However, the distinction lies not merely in the information collected, but in how AI processes that information. Traditional analytics were largely descriptive, reporting metrics such as how many visitors accessed a website, which pages they viewed, and how long they remained on each page. AI systems, by contrast, analyze that same information to generate inferences¹⁸, create consumer profiles, and engage in profiling¹⁹ activities that are regulated under the CCPA and its implementing regulations. Likewise, the GDPR recognizes profiling²⁰ and automated decision-making as distinct forms of processing²¹ subject to regulatory requirements²².

In other words, AI processes information that was historically used for descriptive analytics into information that supports predictive and inferential processing, making such information more readily linkable to an identifiable individual and generating new inferences that may themselves constitute personal information under the GDPR and the CCPA. As a result, privacy compliance under the GDPR and CCPA becomes a far more immediate and consequential consideration than in the pre-generative AI era.

Additionally, the California Privacy Protection Agency has adopted regulations governing certain uses of automated decisionmaking technology, including profiling, which impose additional notice and consumer rights obligations when applicable.

Has the Startup’s Legal Role Changed?

AI also affects the analysis of a startup's legal role under both statutes. As discussed above, the GDPR and the CCPA assign obligations according to an entity's role in processing personal information. Importantly, a startup may occupy multiple roles simultaneously. For example, a company may act as a Business with respect to information it collects from its own customers while simultaneously acting as a Service Provider or Processor when processing another company's customer data. Similarly, a startup that falls outside the CCPA's definition of a Business because it does not satisfy the statute's revenue or processing thresholds may nevertheless become subject to the CCPA when acting as a Service Provider, Contractor, or Third Party on behalf of a CCPA-covered Business.

Consider a Georgia startup that provides AI-powered document summarization services to enterprise customers nationwide. The startup itself generates less than one million dollars in annual revenue and does not independently qualify as a Business under the CCPA. Its first customer, however, is a multinational corporation with substantial operations in California that qualifies as a CCPA-covered Business. Assuming the parties have entered into contracts satisfying the applicable statutory requirements and the startup processes the information only on the customer’s behalf, the startup may act as a Service Provider under the CCPA and a Processor under the GDPR.

Now add one fact. Rather than deleting the customer information, the startup retains it and its customer prompts, uploaded documents, or conversation histories to train its own large language models, fine-tune existing models, develop embeddings, optimize future products, or create proprietary datasets. The startup has introduced a new purpose for processing the information that extends beyond providing services to its customer. That factual change may affect whether the startup continues acting solely on behalf of its customer or instead determines its own purposes for processing, potentially altering its legal role and the obligations that follow. Accordingly, founders should carefully evaluate whether the legal role their startup occupies with respect to each processing activity, rather than assuming they are merely providing services on behalf of another company.

Is the Personal Information adequately anonymized and deidentified?

Finally, AI systems increasingly challenge traditional assumptions regarding anonymization and deidentification because information that once appeared anonymous may become reasonably identifiable when analyzed alongside other datasets.

Under the CCPA, personal information does not include “deidentified” consumer information, which is defined as information that cannot reasonably be used to infer information about, or otherwise be linked to, a particular consumer provided that the business that possesses the information follows the statutory requirements²³. Therefore, under the CCPA, there is a technical “deidentification” requirement and also an organizational requirement. An important exception applies to certain reproductive healthcare information, which remains subject to additional restrictions unless it is used only for limited statutory purposes, retained only in aggregated or deidentified form, and not sold or shared²⁴.

Similarly, under the GDPR, data protection does not apply to anonymous information, namely information which does not relate to an identified or identifiable natural person or to personal data rendered anonymous in such a manner that the data subject is not or no longer identifiable²⁵.

However, under the GDPR, simply substituting the personal information with another identifier is not sufficient to render the information anonymous. For example, replacing the customer “John Smith” with “Customer 49218” does not anonymize the data because the customer number can still be linked back to John Smith. The GDPR expressly treats such information as pseudonymized²⁶ rather than anonymous. It expressly provides that personal data that has undergone pseudonymization but can still be attributed to a natural person through the use of additional information remains personal data²⁷.

Therefore, AI has not changed the legal standards and exceptions governing anonymization or deidentification under the GDPR or the CCPA. Rather, it has changed the factual analysis used to determine whether those standards are satisfied.

Conclusion

As AI systems become increasingly integrated into startup products and services, founders should stop thinking about privacy compliance as something that occurs only when information is collected. Instead, every AI-enabled processing activity should be evaluated independently because each may introduce a new processing purpose, alter the startup's legal role, and trigger additional obligations under existing privacy law. Founders who continue to analyze privacy compliance through the lens of data collection alone risk overlooking the new processing activities and purposes that determine their obligations under existing privacy law.

AI doesn't stop with the Privacy Obligations under the GDPR or CCPA. AI-specific laws such as the EU AI Act and Colorado AI Act may impose additional governance obligations depending on how your AI systems are used.

Notes

1. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data and Repealing Directive 95/46/EC (General Data Protection Regulation) art. 4, 2016 O.J. (L 119) 1 (Personal Data is defined as “information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person. Personal data does not include “anonymous information”, namely information which does not relate to an identified or identifiable natural person or to personal data rendered anonymous in such a manner that the data subject is not or no longer identifiable.”) ↩
2. Cal. Civ. Code § 1798.140 (Personal Information is defined as “information that identifies, relates to, describes, is reasonably capable of being associated with, or could reasonably be linked, directly or indirectly, with a particular consumer or household. Personal information includes, but is not limited to, the following if it identifies, relates to, describes, is reasonably capable of being associated with, or could be reasonably linked, directly or indirectly, with a particular consumer or household. Personal information does not include deidentified or aggregate customer data.”) ↩
3. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data and Repealing Directive 95/46/EC (General Data Protection Regulation) art. 3, 2016 O.J. (L 119) 1. (“(1) This Regulation applies to the processing of personal data in the context of the activities of an establishment of a controller or a processor in the Union, regardless of whether the processing takes place in the Union or not; (2) This Regulation applies to the processing of personal data of data subjects who are in the Union by a controller or processor not established in the Union, where the processing activities are related to (A) the offering of goods or services, irrespective of whether a payment of the data subject is required, to such data subjects in the Union; or (B) the monitoring of their behavior as far as their behavior takes place within the Union; (3) This Regulation applies to the processing of personal data by a controller not established in the Union, but in a place where Member State law applies by virtue of public international law.”) ↩
4. Practical Law Data Privacy & Cybersecurity, Determining the Applicability of the GDPR, Practical Law Practice Note No. W-003-8899 (Thomson Reuters), https://uk.practicallaw.thomsonreuters.com/w-003-8899. ↩
5. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data and Repealing Directive 95/46/EC (General Data Protection Regulation) recital 23, 2016 O.J. (L 119) 1 ("In order to determine whether such a controller or processor is offering goods or services to data subjects who are in the Union, it should be ascertained whether it is apparent that the controller or processor envisages offering services to data subjects in one or more Member States in the Union. Whereas the mere accessibility of the controller's, processor's or an intermediary's website in the Union, of an email address or of other contact details, or the use of a language generally used in the third country where the controller is established, is insufficient to ascertain such intention, factors such as the use of a language or a currency generally used in one or more Member States with the possibility of ordering goods and services in that other language, or the mentioning of customers or users who are in the Union, may make it apparent that the controller envisages offering goods or services to data subjects in the Union."). ↩
6. Practical Law Data Privacy & Cybersecurity, Determining the Applicability of the GDPR, supra note 4. ↩
7. Id. ↩
8. Cal. Civ. Code § 1798.140(g) (defining “Consumer” as a natural person who is a California resident, as defined in Section 17014 of Title 18 of the California Code of Regulations, as that section read on September 1, 2017, however identified, including by any unique identifier.) ↩
9. GDPR art. 4(7) ↩
10. GDPR art. 4(7) ↩
11. Id. ↩
12. Cal. Civ. Code § 1798.140. ↩
13. Cal. Civ. Code § 1798.140. ↩
14. Id. ↩
15. Id. ↩
16. Cal. Civ. Code § 1798.140 ↩
17. Id. ↩
18. Id. (defining personal information to include… “Inferences drawn from any of the information identified in [the] subdivision to create a profile about a consumer reflecting the consumer's preferences, characteristics, psychological trends, predispositions, behavior, attitudes, intelligence, abilities, and aptitudes.”) ↩
19. Id. (defining "profiling" as "any form of automated processing of personal information . . . to evaluate certain personal aspects relating to a natural person and in particular to analyze or predict aspects concerning that natural person's performance at work, economic situation, health, personal preferences, interests, reliability, behavior, location, or movements"). ↩
20. GDPR art. 4(4) “‘profiling’ means any form of automated processing of personal data consisting of the use of personal data to evaluate certain personal aspects relating to a natural person, in particular to analyse or predict aspects concerning that natural person’s performance at work, economic situation, health, personal preferences, interests, reliability, behaviour, location or movements.” ↩
21. GDPR art. 4(2) “’processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction.” ↩
22. GDPR art. 22. ↩
23. Cal. Civ. Code § 1798.140 ↩
24. Cal. Civ. Code § 1798.145(a)(2)(A), (B) ↩
25. Regulation (EU) 2016/679 of the European Parliament and of the Council, recital 26, 2016 O.J. (L 119) 1. ↩
26. GDPR art. 4(5) “Pseudonymisation” means the processing of personal data in such a manner that the personal data can no longer be attributed to a specific data subject without the use of additional information, provided that such additional information is kept separately and is subject to technical and organisational measures to ensure that the personal data are not attributed to an identified or identifiable natural person). ↩
27. GDPR recital 26. ↩

This article is general information, not legal advice, and reading it does not create an attorney-client relationship. Every business is different. If any of this touches your situation, start a conversation.