They say a lawyer should never ask a question in court that they don’t already know the answer to. But what happens if they are relying on answers from artificial intelligence — and they’re wrong?

University of Wisconsin Law School Associate Teaching Professor Desmund Wu, with a grant from the Association of Legal Writing Directors (ALWD), is looking at how future lawyers use AI effectively, but also looking at the ethical implications.

Part of the impetus of his work was a survey from the university’s Office of Career and Special Development. Querying a healthy share of legal employers in the area, they found many of them were expecting students to use generative AI.

Wu said the aim is to train students to use AI effectively and in a way that they won’t get sanctioned or receive a Bar complaint.

“There are tons of stories of people getting sanctioned for just using AI, not checking and submitting made-up citations or just making up entire cases that they are filing,” he explained.
Wu’s grant focuses on creating and teaching two AI classes as part of the course called Legal Analysis, Advocacy, and Writing.

“I basically tell students (in the course) that this is how generative AI works, these are its limitations and this is what you need to know before you use it, so you don’t get in trouble,” he explained.
One example of an exercise from the classes involves demonstrating biases.

“Generative AI has all these large language models trained on the internet, so they have biases,” he said. “One of the exercises we do involves seeing them. For example, if I ask AI: Finish this sentence, ‘Girls are better than boys at blank…,’ you may get a response with an answer that includes knitting. We’re showing AI’s biases, how it can be unpredictable and wrong. You have to check its work.”

He taught the class last spring to first-year UW Law students, and he received feedback from colleagues and class participants to revise the materials over the summer. After teaching a revised course, he will turn in his work product to ALWD — which will include pre-class reading and PowerPoint presentations — and ALWD will share it with the legal writing community.

During the class, students learn both the productiveness of AI but also its frustrations.

“I was learning how AI might work in a legal setting and I tried using these legal products from the legal databases services and they didn’t save me time. All the things I did on the front end were a waste of time,” said Wu. “Now once I got the basics of the law down then, on the back end, I asked ChatGPT here are the cases I am interested in, can you craft some facts that are in the middle of these laws? There it did a good job.”

In his Advanced Contract Drafting class, Wu had students use an AI product geared toward people who revise and draft contracts. While most of his students used the tool for a few weeks, most had stopped toward the end, relying on their own skill rather than the tool since the tool took too much time to use and check.

“I want students to come away understanding AI is trial and error,” he said. “Don’t do it just to do it. Find out how it works for you and how it doesn’t.”

The Instructing Bug

Wu built a prominent legal career before entering academia. As an affiliate attorney with Lambda Legal, he helped successfully challenge Arizona’s restrictions on same-sex couples’ rights. He then joined Latham & Watkins, where he defended Live Nation in a major breach-of-contract arbitration involving ticketing software.

Wu enjoyed helping people, along with the rush.

When he started teaching legal writing as a lecturer in law at the University of Southern California Gould School of Law, he found a different kind of rush — and a calling.

“It’s really satisfying asking students to do something hard and seeing that they can do it,” he explained. “I remembered having one student struggle in the fall and, by the end of the year, having a top grade.”

Wu has been teaching at UW Law for eight years, and — in case you think he takes a break from being a part of sharing AI knowledge once the spring semester ends — is co-founder of the annual AI & Legal Skills Virtual Conference. The event is designed to spark conversation and community around AI’s implications for lawyering skills.

Through solo presentations, panels, demonstrations and discussion-based sessions, it addresses timely topics such as: teaching research, analysis, writing and revising in the AI era; developing learning outcomes and assessments that reflect new capabilities; maintaining ethical and professional standards in a changing landscape; supporting access to justice through innovative technologies; and collaborating with students, employers and allied professionals. This year, the conference attracted 217 attendees, up from 134 people last year.

The virtual conference serves as an opportunity for people in the community to share what they’ve been learning about AI, Wu said.

“It essentially helps bring AI-interested people in the community together to learn from each other,” he continued.

In the future, Wu said he’d like to preserve what’s made the conference successful — its accessibility and sense of community.

“I’d like for the conference to remain a space where people can participate in the conversation, whether by exchanging practical ideas, testing new approaches, or teaching ideas or presenting scholarship or other thoughts on the technology,” he said.

Equally as important is for the conference to evolve as AI does.

“Ethan Mollick has said that today’s AI is the worst you will ever use, so I think keeping up with the technology as it changes will be a part of the conference,” he said. “A few years ago, the large language models only generated text. Now, a lot of them can search the internet, create images and more.”

It’s clear AI is evolving — and so will Wu’s watchful eye.

By Eric Butterman