“In the end, AI’s high-end technology will be used for censorship and surveillance; the low-end applications will be used for soft porn and clickbait.” — a Chinese netizen.

Orwell foresaw Big Brother’s future a century later, long before computers were commonplace. In 1984, the Ministry of Truth invents a “novel-writing machine” — essentially a conceptual prototype of generative AI. Prompts in the novel are called “political parameters.”

Feed in the instructions and the machine, using fixed formulas and modules, mechanically assembles the entire plot — beginning, development, turn, and resolution — stuffing it with Newspeak, vulgar titillation, or the required political propaganda. It rapidly spews out crude text. No human creativity or feeling is required.

Those who choose to work in this industry might as well all be called Julia. In 1984 she works in the Fiction Department, operating and maintaining the novel-writing machines, smoothing the places where the mechanical prose doesn’t flow, and checking for political taboos — what we now call RLHF and alignment.

Julia specializes in pornographic literature (Pornosec), what China calls soft porn. This spiritual opium is used to drain the proles’ surplus energy and discontent so they stay addicted and never have time to form thoughts of resisting the Party. Men are barred from the workshop; the operators are all young, unmarried women. The Party believes young women are more easily confined by pure discipline and mechanical labor than men, and less likely to develop heretical ideas.

The irony is that Julia herself loves to read. She knows perfectly well that this mass-produced literature is garbage entertainment. To escape the dull production line she rents a room on the second floor with no telescreen — the equivalent of a safe house with no smart-home devices watching you — seeking emotional and physical freedom.

She does not know the landlord is a senior Thought Police officer. Behind the church painting on the wall is a hidden telescreen. The Party deliberately rents out such rooms to trap potential thought criminals. Big Brother will not allow a love higher than loyalty to the Party.

Julia is arrested and sent to the Ministry of Love for torture and remolding. No one knows what she saw in Room 101. When she next sees daylight she has become a numb stone.

The novel-writing machine and the telescreen are complementary machines. The first dumbs people down; the second catches those who failed to be dumbed down. In the final utopia of Big Brother’s blueprint, the telescreens will be dismantled one by one. They will no longer be needed.

点击图片查看原图