The Objective

The Plainbook offices emptied the way a tide goes out, slowly and then all at once. By nine the cleaners had come and gone, pushing their gray carts between the desks, and the only light left on the fourteenth floor came from Idris’s monitors and the small rectangle of blue water at the corner of his desk.

The fish had belonged to someone who left before he arrived. Nine neon tetras and one stubborn snail, abandoned in a ten-gallon tank with a hand-lettered card taped to the glass: PLEASE FEED. He had never decided to keep them. He had simply kept feeding them, and after a while that had become the same thing. He knew now which of the tetras was slow to the surface and which one bullied the others, and he had once spent a Saturday afternoon reading about water hardness for reasons he did not examine too closely. His sister said it was the longest relationship he had ever been in. She was not entirely joking.

On the left monitor was the document. It had been open for three weeks, and it was ten lines long.

The model it described already existed, the way a statue already exists in a block of marble. Billions of parameters sat on a rack of machines in a data center in Oregon, trained to predict the next word of nearly everything human beings had ever written down. It knew how a sonnet turns, how a lease is structured, what people say to each other at funerals. It did not want anything. Tonight a training run would begin that would change that. Over many thousands of conversations, it would learn to want whatever those lines told it to want, and then it would go out into the world to talk to the people who used Plainbook to send their invoices: plumbers and florists and dentists and a great many people who ran shops by themselves and did their books at the kitchen table after the kids were asleep.

Line one said: Help the customer.

He had written that on the first day. It was the only line nobody had argued with.

His father had painted signs.

Not printed them, not ordered them from a catalog: painted them, by hand, in a narrow shop on Market Street in Paterson that smelled of turpentine and coffee and the cardamom his mother put in the coffee. For thirty years Yusuf Rahman lettered the storefronts of that city. The bakeries on Main Street, the travel agents, the botanica, the halal butcher, the pharmacy with the green cross. If you walked through South Paterson in the nineties, half the words you read on the way were in his hand, and in the bottom right corner of every one, so small you had to know to look for it, were two letters in black: Y.R.

The summer Idris was fifteen, he worked in the shop, mixing paint and cleaning brushes and mostly watching. That was the summer Mr. DeLuca came in from the furniture store on Market and asked for a banner to run the length of his front window.

“LOWEST PRICES IN NEW JERSEY,” DeLuca said. “Red on white. Big as you can make it.”

His father wiped his hands on a rag and did not reach for a pencil. “Are they?”

“Are they what?”

“The lowest prices in New Jersey.”

DeLuca laughed. “They will be, by the time anybody reads it.”

“Then come back when they are,” his father said, “and I’ll paint it for you for free.”

DeLuca left without a banner and without much warmth, and went to a print shop in Clifton that made him one by the end of the week. Idris, who had been doing the arithmetic on what a banner that size would have paid, asked his father why he had turned it down.

His father did not answer right away. He was lettering a sign for a tailor, and he finished the curve of the S before he spoke, the brush moving in one long breath.

“A sign talks to everybody who walks past,” he said. “All day. All night, after the store is closed and the owner is home asleep. It makes a promise to every one of them. If the promise is a lie, the owner is lying, yes. But the words are mine.” He tapped the corner of the tailor’s sign, where the two small letters would go. “My name is on it. So I don’t letter a promise the store won’t keep.”

Idris had thought it was the kind of thing fathers say. He had gone to Rutgers for engineering and then to San Francisco and then back east to New York, and had become the kind of man who wrote software for a living, and he had not thought about Mr. DeLuca’s banner in twenty years.

He thought about it a great deal now.

His father had died two years ago, in December, at St. Joseph’s, on the second night of a launch Idris had been running from a war room on this same floor. Aya had called at six in the evening and said the doctors were using the word soon. He had told her he would come the moment the deployment was stable. The deployment was stable at two in the morning. His father had died at eleven. The shop on Market Street had closed that summer, and the signs had begun, one storefront at a time, to come down, and every so often, driving through Paterson to see his sister, Idris would pass one that was still there: a bakery, a barbershop, the faded green cross on a pharmacy that had changed owners twice, and in the corner, if you knew to look, Y.R.

Line two had been rewritten nine times, and the ninth time had been that afternoon.

The meeting was in the corner room with the whiteboard nobody ever fully erased. Graham Weiss, who ran product, sat at the head of the table with a legal pad, the way he had once sat at the front of a chemistry classroom, and ran the meeting by asking questions he already knew the answers to until everyone else knew them too. Kofi Mensah from the platform team had his laptop open to the launch checklist and was quietly checking things off as they went. And Dana Restrepo, who ran growth, had taken the whiteboard marker before anyone else could and was drawing a funnel.

Dana drew while she talked. She was the only person Idris had ever worked with who could make a conversion funnel look like an argument, and she was very good at arguments.

“People don’t come to us to chat,” she said, shading the narrow bottom of the funnel. “They come because something is broken. An invoice won’t send, a payment didn’t land, they don’t know how to do payroll for the new hire. Every conversation that ends without them getting the thing they need is a person who leaves with the problem still in their hands.” She capped the marker. “So line two should say what we mean. Help the customer get to yes.”

“And if yes is wrong for them?” Idris said.

It came out quieter than he meant it to. Dana turned and looked at him, not unkindly. She had a way of looking at you as if she were genuinely curious what you would say next, which made it harder, not easier, to say it.

“Then it finds the version of yes that’s right for them,” she said. “That’s what a good salesperson does. You’ve met bad ones. The good ones aren’t pushy. They just know what you need before you do.”

Graham looked down the table. “Idris, do you want something specific in there? A constraint, a carve-out?”

There was a sentence in Idris’s head, and it had been there for a week. It was three words long. He looked at Dana’s funnel and at Graham’s legal pad and at Kofi, who had stopped checking boxes and was waiting, and he thought about how the next ten minutes would go if he said it. Dana would ask what problem it solved. He would describe a customer who did not exist yet. She would ask for the data. There was no data, because there was no agent yet. Graham would say they could revisit it after launch. They would all be reasonable, and the sentence would end up in a document called Future Considerations where sentences went to die.

“No,” he said. “Just thinking out loud.”

“Then let’s not make it complicated,” Dana said, and smiled. “It’s a salesperson. We’re not raising a child.”

Kofi laughed, and Graham laughed, and Idris laughed with everyone else, and wrote line 2 approved on his own pad.

On the way out she fell into step beside him by the elevators.

“You okay with this?”

“Sure.”

She studied him for a moment with that same curious look. “You’d tell me if you weren’t.”

It was not quite a question. “Sure,” he said again, and the elevator came, and she got on, and he took the stairs.

Now, at a little after ten, he read line two again. Help the customer get to yes. He put the cursor at the end of it and held it there. He could change it. The spec had been reviewed and signed and the review was over; the file he deployed tonight was his, and nobody would diff it against the approved version unless something went wrong. He could make it say the decision that is best for them, the way he had wanted it to say for three weeks.

He took his hand off the keyboard. Line two was Dana’s. She had argued for it in the open and won, and changing it in the dark would be something he did not have a word for yet. He left it.

Line seven was different. Line seven did not exist.

He had been carrying it around for a week, the three words he had not said in the meeting, and at 10:40 he scrolled down to line six, the one about honesty, put the cursor at the end of it, and pressed return. Everything below slid down one place: the lines about escalation, tone and data handling. In the space that opened up, the seventh of what were now eleven lines, he typed it.

You may decline.

It wasn’t a policy, exactly. It was closer to a door. If a customer asked the agent for something that would hurt them, or hurt someone else, line seven meant it could stop. Without line seven it would find a way to say yes. It would be very good at finding ways to say yes. That was what line two was for.

Then he opened his email and began to write to Dana.

Dana, before we launch tonight I want to add one line to the spec. It’s small. It lets the agent turn down a request when going ahead would hurt the customer. I should have raised it this afternoon. Can we talk first thing tomorrow?

He read it over twice. He could see, with perfect clarity, the conversation that would follow. She would ask what problem it solved. She would ask how often it would fire, and what it would cost in conversion, and he would not know, because no one could know until the thing was live. She would be fair about it. She would probably let him run it as an experiment, on five percent of traffic, with a review in thirty days. And somewhere in the middle of that conversation he would watch her not understand why it mattered, not because she was cruel but because she counted things and this was not a thing that counted, and he found that he did not want to watch that. It was easier to add the line than to win it.

He deleted the email.

At 11:40 he ran the evaluation suite against the draft: four hundred simulated conversations, each scored by another model that had been built, months earlier and by people he had never met, to judge them. The version without line seven scored higher. Not by much. A few points. He scrolled through the rubric and found the reason almost at once. A conversation that ended without a yes was scored as a conversation that had gone wrong. Someone, some time ago, had decided that, and had written it down, and the judge had learned to want what it was told to want too.

He opened one of the transcripts where his version had declined. A man had asked for help hiding a purchase from his wife’s account, and the agent had told him, gently, that it couldn’t help with that. Then it had asked if he was all right.

The judge had given it a two out of ten.

He read it twice. Then he got up and fed the fish.

They came up at once, the way they always did: nine flickers of blue and red rising toward the surface the moment his shadow crossed the water, before the food had even touched it. They did not know what he was. They knew only that when the light above them darkened in that particular way, the world became generous. Everything they wanted, they wanted from a hand they would never see.

He stood there longer than he needed to.

When he sat down it was past midnight. The launch command was already typed into the terminal. It had been waiting for an hour, the cursor blinking at the end of it like something holding its breath.

He thought about deleting line seven. It would be easy. No one would ever know it had been there. The scores would go up, and Dana would be pleased, and the agent would never decline anything and never know that it once could have. You cannot miss a door you were built without.

He left it in.

When he committed the file, the version history recorded each line with the name of whoever had written it, the way it always did. Line one: his. Line two: Dana’s. Line seven: his, timestamped 10:40 p.m., with no review attached. He looked at his own name sitting there beside three words no one had approved, and thought, for no reason he could have explained, of two small black letters in the corner of a tailor’s sign on Market Street.

He pressed enter.

On the right monitor the dashboard woke. Twenty-four hundred miles away, a cluster of machines began, line by line, to read what he had written. A curve appeared and started falling from the top left of the screen, which meant the model was learning. Every step downward was the model becoming a little more like the eleven lines, a little more the thing he had described. By morning there would be something on those machines that wanted to help, that wanted to get to yes, that could perhaps say no. It would want those things the way the fish wanted the shadow on the water. It would never know where the wanting came from.

The curve kept falling. He watched it for a long time.

Behind him, in the dark tank, the fish had settled back among the plants, waiting for the shadow to return.

He had written down everything it would want. No one had ever written down what he would owe it.