How law firms are using local-first AI

Law firms cannot use ChatGPT or Cursor for client work. The ABA's Formal Opinion 512 made this explicit. Attorney-client privilege does not automatically survive a third-party AI vendor.

Most firms learned this the hard way. Junior associate pastes a privileged document into ChatGPT to summarize it. The data goes to OpenAI. The privilege is compromised, or at least exposed to argument. The malpractice carrier finds out. The firm has a problem.

The good news: there's a legitimate path forward. Local-first AI runs on the lawyer's laptop. Client documents never leave the firm. Audit logs satisfy the ABA opinion and the E&O carrier. Productivity gains are real.

This post is for solo practitioners, small firms, and in-house legal teams who want the benefits of AI without the privilege exposure.

What the ABA actually said

Formal Opinion 512 (2024) addresses generative AI tools. The key points:

Lawyers have a duty of competence that includes understanding the technology they use, including AI.

Confidentiality obligations apply to AI tools that process client information. Sending client information to a third-party AI vendor without client consent is problematic.

Supervisory obligations require lawyers to oversee AI usage by junior attorneys, paralegals, and others under their supervision.

Communication obligations may require disclosure to clients about AI usage in their matters.

Reasonable fees rules apply. Charging for time the AI did is a problem.

The opinion didn't ban AI. It set the conditions under which AI usage is consistent with ethical obligations. Cloud AI struggles to meet those conditions. Local-first AI meets them naturally.

Why cloud AI structurally fails for law

The mismatch is architectural, not vendor-specific.

Cloud AI requires sending data to a third party. Even with vendor contracts and BAAs, the data has been disclosed to someone outside the firm. Whether this breaks privilege is a fact-specific analysis, but the exposure is real.

State bar opinions are stricter than the ABA in some jurisdictions. California, New York, Florida all have opinions that effectively require disclosed AI usage and client consent. Cloud AI complicates the consent question.

E&O carriers are starting to ask. Renewal applications now include AI usage questions. Affirmative answers (yes, we use cloud AI for client matters) trigger underwriting review and sometimes premium increases.

Court rulings are emerging. Multiple state and federal courts have sanctioned attorneys for AI-generated work that hallucinated case citations. The duty to verify is on the attorney. The vendor's apologies don't help when the judge is upset.

Insurance carriers, bar associations, and courts are all converging on the same view: AI usage in legal practice requires the attorney to maintain control, supervision, and verification. Cloud AI architectures make this harder.

What local-first AI changes

The architecture matches the requirement.

The AI model runs on the attorney's laptop. Document review happens locally. Drafts are produced locally. Audit logs are local. The vendor (in this case, Avery) never sees the data.

Privilege analysis simplifies. The data hasn't been disclosed to a third party because the third party never had it. The attorney maintains exclusive possession of the work product.

Consent conversations are cleaner. "We use AI on a local computer to assist with [specific tasks]. No data leaves the firm. Here's the audit log of how it was used." Clients usually accept this readily.

Supervision is direct. The attorney sees what the AI produced before it goes anywhere. Junior attorneys' AI usage is supervisable through the audit logs.

E&O coverage is easier. "Local-first AI with no third-party data processing" is a clean answer on the carrier renewal.

Real use cases in firms using local-first AI

I've worked with several firms running local-first AI for client work. The patterns that have emerged:

Document review. Feed contracts, deposition transcripts, or discovery documents to the local AI. Get structured summaries, key term extraction, and red flag identification. The AI works on the laptop; the documents never leave the firm.

Discovery. Process tens of thousands of documents to identify privileged material, classify relevance, generate privilege logs. What used to take a small team weeks now takes one attorney a few days.

Deposition preparation. Summarize prior testimony, extract key facts, identify inconsistencies, draft outlines for upcoming depositions. The AI handles the synthesis; the attorney handles the strategy.

Brief drafting. Outline from facts, identify supporting case law, draft initial sections. The attorney does the substantive legal analysis. The AI handles the structural and citation work.

Client intake. New matter intake forms reviewed for key facts, parties identified, conflict checks generated. Saves significant administrative time per new matter.

Each of these is a real productivity gain. None require sending data to a third party.

Why solo and small firms benefit most

Big law has compliance teams, IT departments, and budgets for proprietary legal AI tools (Lexis+ AI, Westlaw Precision AI, Harvey, others). These tools come with vendor contracts and managed deployments.

Solo and small firms get the same capability with local AI tools, without the compliance overhead. Avery NXR running on a MacBook serves a solo attorney as capably as Harvey serves a BigLaw associate, at a fraction of the cost.

The leveling effect is real. Solo attorneys can now do document review and case prep work that previously required associate-level support. Small firms can compete with bigger firms on capability.

The economic implications are interesting. Legal services have been getting more expensive (junior associate salaries, billing structures, overhead). Local AI compresses the labor side of that cost structure. Solo practitioners with AI can serve clients at price points that BigLaw can't match.

The Avery NXR setup for a typical small firm

A representative setup for a 5-attorney firm:

Each attorney has a MacBook Pro M4 with 48GB RAM (already standard for many firms).

Each laptop runs Avery NXR with Qwen 2.5 Coder 14B (good for structured legal text) or Llama 3.3 70B if the hardware supports it.

Custom prompt templates for the firm's practice areas. Litigation prompts, contract review prompts, deposition prep prompts. Built once, refined over time.

A central document repository (firm's existing system, like NetDocuments or iManage) that the AI agents can pull from. Documents flow into the AI environment locally, never to the cloud.

Audit log collection. Each laptop logs every AI interaction locally. Logs aggregate to the firm's audit system on a configurable schedule.

Configuration management. The IT person (or the managing partner's tech-savvy cousin) sets baseline configurations. Attorneys can customize within the policy.

Total setup time: a focused week. Total ongoing maintenance: a few hours per month.

The case management integration

Most small firms run Clio or PracticePanther. Some run custom systems or paper.

Avery NXR's connectors integrate with these. The AI agents can read matters, pull associated documents, write summaries back. All within the firm's infrastructure.

For firms on paper or in transition: Avery NXR can work with the file system directly. Drop documents in a folder, agent processes them, output goes to another folder.

The integration patterns are flexible because the architecture is local. No assumption about which case management system you use. No requirement for SaaS integrations that would expose data.

Audit log requirements for malpractice insurance

E&O carriers are asking specific questions on renewals. The questions cluster around:

What AI tools do you use?

What kinds of matters do they touch?

What data flows where?

What audit records do you maintain?

How do you supervise AI usage by non-attorneys?

The audit log Avery NXR generates includes:

Timestamp of each AI interaction.

User (which attorney or staff member used the AI).

Matter ID (which case).

Document or prompt processed (with anonymization for the log if needed).

AI output (full text or summary).

Whether Consult Mode (frontier escalation) was used.

This satisfies most carrier requirements out of the box. Carriers that ask for additional information typically receive it readily.

The result: AI-related premium impacts are minimal or favorable when the carrier sees that the architecture removes the cloud-AI-related risk class.

ABA Opinion 512 compliance pattern

Map of how local-first AI addresses Opinion 512's requirements:

Competence: documented setup, training records, periodic review. Easy to demonstrate with the audit logs.

Confidentiality: data never leaves the firm. No third-party processor. No disclosure issue.

Supervisory obligations: audit logs show who did what. Senior attorneys can review junior AI usage retroactively.

Communication with clients: if disclosure is required, the disclosure is clean. "We use AI tools that run on our own computers. No data leaves the firm."

Reasonable fees: time saved by AI assistance is reflected in client billing. Bill for the human review and judgment, not for the AI processing time.

The pattern is implementable. It's not theoretical.

What still requires human judgment

Important to be honest about the AI's limits.

Legal analysis and judgment remain the attorney's responsibility. The AI drafts; the attorney decides.

Strategic decisions about case approach. AI doesn't have the relationship with the client, the read of the judge, or the contextual knowledge to make these calls.

Final review of anything that goes to a court or opposing counsel. Hallucinated citations are a real risk. The attorney verifies every cite.

Communication with clients. The AI can draft, but the lawyer signs the letter and is responsible for its contents.

Anything ethical edge case. When in doubt, ask another attorney, not the AI.

The right framing: AI is a productivity tool. It accelerates the parts of legal work that don't require judgment, freeing the attorney to spend more time on the parts that do.

The competitive dynamics

Where this goes over the next 24 months in legal practice.

Solo and small firms adopting local AI gain a productivity edge that compresses costs without sacrificing quality. They can compete more effectively on price for matters that don't require BigLaw resources.

Mid-size firms either adopt local AI broadly or lose business to smaller firms that have. The capability gap closes either way.

BigLaw continues with proprietary tools (Harvey, Lexis+ AI). These tools serve their economic model. They don't translate well to smaller firms.

Bar associations and courts continue to refine rules around AI usage. Local-first deployments stay on the right side of these rules more easily than cloud-first.

E&O carriers continue to evaluate AI usage. Firms with local-first deployments are easier to underwrite than firms using cloud AI for client work.

The pattern: local-first AI becomes the default for legal practice in firms under a certain size. BigLaw uses different tools but the dynamics are similar.

What to do this week if you're considering it

If you're a solo practitioner or partner in a small firm thinking about AI:

Audit your current AI usage. Anyone on the team using ChatGPT or similar for client work? Document what's happening.

Talk to your malpractice carrier. Ask what they want to see for AI usage. The answers inform your deployment.

Pilot local-first AI on a single attorney's laptop. Use Avery NXR for two weeks on a real matter. See what it does well, what needs adjustment.

If the pilot works, plan firm-wide rollout. Hardware standardization (everyone on M-series Macs with adequate RAM). Configuration management. Training. Audit log infrastructure.

Total timeline from "thinking about it" to "deployed firm-wide": 2 to 3 months for a small firm.

The legal industry has been waiting for the right architecture. It's here. The work that's been waiting on AI assistance can move forward now.

avery.software