Value Alignment Toolkit for Cowork

Examples

Before any skill is trusted on an actual legal department, it is tried against three invented ones - built to be different from each other on purpose.

Why pretend legal departments

To check that a skill behaves well, it has to be run against something. Running it against an actual client the first time would mean finding out about a mistake only after a general counsel had already seen it. Instead, each skill is exercised against a handful of invented legal departments first - imaginary companies, with imaginary people, documents, and numbers, built specifically to stand in for an actual one.

In plain terms: in software testing, an invented scenario built to stand in for an actual one is called a test fixture. None of it exists - no company, no general counsel, no numbers - but it is built to be realistic: the kind of documents, gaps, and messiness a working legal department would actually hand over. Running a skill against a test fixture lets mistakes get caught and fixed before an actual department ever sees the output.

The three fixtures used here are called Meridian Diagnostics, Northwind Materials, and Verba. None of them exists. Every company name, person, number, and quotation attached to them was made up for this purpose, and none of it is drawn from Thomson Reuters material or from any actual organization. Building three, rather than one, is deliberate: actual legal departments do not all arrive in the same condition, so a skill only proves itself once it has been tried on more than one starting point.

Three starting points

Legal departments range from highly organized to barely organized, and everything in between. Each fixture below was built to represent one recognizable point on that range, so a skill can be checked against each in turn.

Clean input

Meridian Diagnostics

A mid-size, regulated medical-device company with an established, well-staffed legal department. Its metrics and business objectives are already written down in clean, formal documents, tracked through dedicated systems, and reported on a regular schedule.

What it represents: the "happy path" - a department that arrives with good material already in hand. This fixture checks that a skill makes proper use of solid input, rather than needlessly redoing or second-guessing work that is already trustworthy.

Empty / starting from scratch

Northwind Materials

A privately held manufacturer whose very first General Counsel has just been hired. There is no metrics program, no tracking system, and nothing written down about what the legal department is trying to achieve.

What it represents: the opposite extreme - a department with nothing to build on. Here a skill cannot simply read existing documents; it has to ask the person good questions about the company's goals and help assemble a sensible starting set, without ever inventing numbers or history that were never given to it.

Messy / low-quality input

Verba

A fast-growing software startup with a legal team of just three people and no dedicated operations staff. What passes for its metrics today is a hand-kept spreadsheet with years of ad-hoc additions, a half-finished slide put together from memory for a board meeting, and an email thread about spending.

What it represents: the messy middle, which is where most departments actually sit - something exists, but it is inconsistent, incomplete, and not fully trustworthy. Here a skill has to sort out duplicates and stale entries, set aside what it cannot verify, and say plainly what is still missing, rather than smoothing the mess into a confident-looking result.

Why all three get tested

A skill that only performs well on tidy, complete input is not actually useful to most legal departments, because most legal departments do not show up that way. Some, like Meridian, already have clean documentation. Others, like Northwind, have nothing yet. Most, like Verba, sit somewhere in between with data that is partial and needs a critical eye. Testing against all three is how the skills are checked to hold up across the range, rather than only in the easiest case.

This also reflects the design principle explained on How it works: every result is a labeled first draft that flags its own gaps rather than papering over them - and that has to hold true whether the input is Meridian's clean tables, Northwind's blank page, or Verba's messy spreadsheet. Testing against all three starting points is how that gets checked, rather than assumed.

Where this might be wrong - please check: these three fixtures were chosen to span clean, empty, and messy starting points as a reasonable spread of what TRI sees across actual legal departments. If TRI's experience suggests a different or additional starting condition is common and should be tested, that is easy to add.