Tokens, tokens everywhere

Tokens, tokens everywhere

Where Microsoft, Anthropic and OpenAI started, Legora now follows (and I suspect Harvey will soon as well)

As charges for GenAI move away from flat subscriptions towards per-token pricing, law firms need to grapple with the change

The transition may be a little painful, but consumption based pricing is far better aligned with internal processes than subscriptions

Firms are well versed at capturing billable vs non-billable costs, recording the former against matter files and then either billing out as a disbursement alongside hours worked (or rolled up into fixed fees)

Legora/Harvey now look increasingly like photocopier charges. You know how to manage those 🙂

But let's focus on the short term transition

(Ideally) ahead of any changes to per-token pricing, firms need to be taking a look at existing token usage:

(a) By users - this is the easier part as most vendors track this and will share it if you ask nicely. A small number of super users will almost certainly be responsible for the majority of tokens used. Understanding those use cases, and engaging early with those super users will be key

You're ideally wanting to ensure that the benefits from their use exceed the forecast costs. Alternatively you may want to re-engineer those high token use cases into cheaper solutions ahead of changes to the charging mechanism. Or put a stop to the excessive cat video generation - entirely your call!

(b) By matters - I suspect this is harder and will likely need engagement with vendors to produce the required reports. This may not be available for historic use cases if matter codes were not originally captured

There are two reasons for doing this:

The first is to get tokens accurately recorded against matters (or internal project codes) so that the full cost is understood, and initial views can be taken on how this will impact client billing and pricing

The second is to look at matters thematically to understand areas of high spend (e.g. corporate DD is likely higher than marking up a single employment contract. Other differences may be less obvious). This then gives the firm it's priority areas to start to standardise approach and (again) re-engineer where appropriate

Some firms have had a pretty good grip on this over the past couple of years, others have not

The risk with the latter camp is that you have an unknown exposure to inefficient use of tokens, where the cost of those tokens is going to increase

You can either get ahead of it, or you can wait for the bill to land and deal with it then - no more free lunches I'm afraid

But overall I do think it's positive that economic reality is starting to bite - it will force a level of discipline around GenAI use cases that (if my Linkedin feed is to be believed) has been missing

Tokens, tokens everywhere!