Why ChatGPT Is the Wrong Tool for This Job: Jev vs Laya Explained on One Email

0:00One Email, Three Questions: the classification job

Say you are integrating an LLM into your company, and here is the use case: support emails. This one just landed. Billed twice for March. Refund today, or we cancel. Before a person reads this email, you want the LLM to answer three questions about it. Which department gets it: billing, technical, sales, or other. How urgent is it: not urgent, soon, or blocking. And will this customer cancel: yes or no. You do not want a paragraph back. You want three answers your code can act on, with a number on each saying how sure the model is. So you send the email to a chatbot model, the kind behind ChatGPT. Two new models say that chatbot is the wrong tool for this job. Jev, from TypeSafe, is a paid service. Send it the email and the three questions, and it sends back three answers with a probability on each, in one query, in under half a second. And it never writes a word, because the answers you allow are set before it runs. Laya makes the same promise, and Laya is a free download. One is closed, so nobody outside TypeSafe can see how it does this. The other is open. So we open Laya, and what we find is the only look there is at how a machine like this works. Three answers, a number on each. That is the whole job.

1:49The Chatbot Writes: structured output and token cost

The chatbot answers in words. This looks like a billing issue, the customer sounds upset and may cancel. Your code now has to figure out what that sentence means before it can route anything. So you tighten the prompt and ask for a fixed format: department, urgency, cancel, each with a confidence. The model writes that too, one token at a time. Braces, quotes, the names, the numbers. The format is fixed. The contents are not. The model can write a department that is not on your list, because it is writing, not choosing. So your code has to check every answer before trusting it. The confidence is the same problem. When the model writes a confidence of ninety-five percent, it is predicting the tokens that sound confident. A model that is right ninety-five percent of the time and says nothing about the other five cannot be left alone. So a person still checks. Two smaller costs follow from the same cause. Every token it writes is an output token, about five times the price of an input token, and you pay for the braces. And the tokens come out one after another, with a reasoning model writing hundreds of thinking tokens first. Seconds, sometimes minutes, for three answers. Words you have to check. A confidence you cannot measure. Paid by the word, one token at a time. One cause: the model writes.

3:39Laya Reads, Then Scores: one encoder pass

The chatbot wrote its answer. Laya does the job a different way. It reads, then scores. Open it up. Here is what the model is handed. Your email. Then your three questions, and under each question every answer you allow. Billing, technical, sales, other. Not urgent, soon, blocking. Yes, no. All of it in one line of text. The model reads that whole line once, every word against every other word. That is the entire run. Nothing comes out one token at a time, because nothing comes out at all. Then every allowed answer gets a score: how well does this answer fit this email. For each question, the scores become percentages that add up to one hundred. The result, from Laya's own card. Billing, ninety-four percent. Urgency, blocking. Cancel, eighty-nine percent yes. Thirty-three milliseconds on an old GPU. The first wall from chapter two is gone. The answer can only be one you listed, because the model scored your list and nothing else. Nothing to parse, nothing to check. A fourth question tomorrow? Add it to the line. No retraining. TypeSafe describes Jev the same way: one query, every allowed answer scored at once, nothing written. What is different inside, TypeSafe does not say. The second wall is the number. Can your code trust ninety-four? During training, Laya is shown emails where the true answer is known. It says a percentage, then the answer is revealed. Say ninety and be right only half the time, and it loses points. Say fifty and be right nearly every time, and it loses points too. The only way to keep the points is to say the real odds. One caveat, from Laya's own numbers. Fresh from the download, the model is too confident. You correct it with a few hundred of your own emails where you know the answers. One adjustment, about an afternoon. After that, ninety-four means about ninety-four, and your code can act on it. Above ninety, route the email. Below, send it to a person. The email and every allowed answer were read together, once, and each answer got a score.

6:35Where the Free One Breaks: cold accuracy and label lists

So Laya does the job in one pass, with answers you can act on, for free. Why would anyone still pay for Jev? Laya's own documents say where it loses. First, what Laya wins. Speed: thirty-three milliseconds on your own GPU, against about a quarter of a second for Jev over the internet. Accuracy, on two thousand decisions, including support emails like yours: Laya seventy-seven percent, Jev seventy-three. The first break is that seventy-seven. That is Laya after an afternoon of training on that test's own example emails. The plain download, asked cold, with no training, scores thirty-six percent, barely better than guessing. Jev's seventy-three is cold. So Laya has to be taught your emails first. Jev does not. The second break is a long list of answers. Say your company has seventy-seven departments instead of four. Jev takes a list that long, up to two hundred and fifty-five answers, and still scores eighty-seven percent. Laya drops to forty-three. Its list has limited room. With four departments, every name fits. With seventy-seven, each name gets only three or four tokens, and names like card fee and card declined look the same to the model. Keep your list under about twenty. Laya wins the small, taught job. Jev wins the cold, wide one. Three answers, one pass, no words.

8:28Which One, on Your Email: fine-tune or brief

Back to your help desk. You have the email, three questions, and two models that score instead of write. Which one do you wire in? The verdict. Jev is more reliable and stronger, and it needs almost no work to set up. You send the email, and three honest numbers come back. Laya is free, but you have to train it and work on it to get its accuracy up. Free means the work moves to you. Here is the gap, cold. On support chats, where the model picks the next step out of thirty, Laya untaught gets seven in a hundred right. Jev, untaught, gets sixty-seven. Laya is a small model scoring your list without knowing what your departments mean. And its numbers lie high until you fit one number on emails you know, as in chapter three. Jev's come honest from the service. So what is Laya for? If you have old emails with the answers already known, teach it. A few thousand of them, a few minutes on one GPU, and the same Laya goes from seven to eighty-two. Past Jev. Your emails never leave your machine, and the bill is zero. Jev cannot be taught. Its one lever is what you send with the email. Send the handbook, ten solved examples and the reasons, and Jev climbs too, to eighty-seven. So you brief Jev, or you teach Laya. Either way, the work is yours. Jev is stronger, more reliable, and ready out of the box. Laya is free, if you do the work. Three answers, one pass, no words.