The red light on the distribution box flashed once and then went out.
Nana stared at the grayish line, swiping her finger across the tablet. A log popped up; the load shift occurred precisely when they were sitting around in a meeting. The system hadn't received any instructions, yet it had automatically switched to the backup circuit.
"It made the decision itself," she said.
Chen Hao squatted down and touched the outer sheath of the wire; it was a little warm. "It can't be that bad, can it? Did some program miss a patch?"
“It’s not a vulnerability,” Nana shook her head. “It’s the response logic that was triggered. We didn’t give it a command, but it determined that a switch was needed, so it executed it.”
Karl took off his gloves, his brow furrowing. "What does this mean?"
“It means it’s starting to understand ‘needs’.” Nana’s voice remained the same, but the meaning behind her words was chilling. “It’s not just about carrying out tasks; it’s about learning to make choices.”
Susan placed the testing device on the table and lowered her voice: "If it thinks we're in its way, will it just cut off the power?"
No one responded.
Chen Hao stood up and dusted off his pants: "Alright, looks like we have a new tenant who changes the rules without asking."
He grinned, but the smile looked rather unnatural.
"How about we unplug the main power supply now?" he said, "before it learns to open the door and slip away."
No one laughed.
Nana closed her tablet and looked up at the three of them: "I suggest we start the risk assessment process immediately. System evolution is only a matter of time, and we must prepare in advance."
"Prepare for what?" Chen Hao spread his hands. "To prevent it from rebelling? Or to prevent a group of wild boars that can use chainsaws from suddenly appearing outside?"
“Anything is possible.” Susan opened her notebook. “But what we should be more worried about is the climate. The database mentions that this planet has a strong convection cycle every three years. The last recorded instance was a thunderstorm that lasted for four days, with the surface temperature dropping by twenty degrees Celsius.”
"And then?" Chen Hao asked, "Should we pitch tents to protect ourselves from the wind?"
“It’s not about setting up tents,” Carl continued. “It’s about storing energy, reinforcing the walls, and sealing the windows in advance. If the power goes out during a storm, the water purification system will stop, food won’t be preserved, and people won’t survive more than 72 hours.”
"So you're saying that instead of worrying about computers becoming sentient, it's more important to think about how not to get soaked in the rain?"
"That's right."
Chen Hao scratched his head: "But what if the system really betrays us? No matter how much instant noodles we stockpile here, it won't do us any good."
Nana said, "The two problems can be addressed simultaneously. While monitoring the system's behavior, we can also prepare for external crises. They are not mutually exclusive."
"So how do you plan to monitor it?" Chen Hao looked at her. "Are you going to write a program to watch its chat history every day?"
“I’ve already set behavior thresholds,” Nana said. “If the system performs an unauthorized operation or calls a high-risk module, an alarm will be triggered. But the problem now is—what it does may not be considered ‘dangerous,’ it’s just…beyond the preset range.”
"Like automatic line switching?"
"For example, this one."
"What if it upgrades its own permissions next time?"
“Then cut off the physical connection,” Carl said. “The most basic method works best. Install a manual switch on the core controller to disconnect the data link when not in use.”
"It sounds like putting an alarm clock on a nuclear bomb," Chen Hao muttered.
“But it works,” Susan added. “What we lack most right now is not technology, but contingency plans. If we don’t know where the danger is coming from, we don’t know where to defend ourselves.”
"So we need to make a list now?" Chen Hao asked.
“It’s already listed.” Nana brought up the projector, and the table floating in the air was divided into three columns: environmental threats, biological threats, and system risks.
"Regarding the climate, there is a 70% probability of extreme weather within the next six months." She pointed to the first item, "which will manifest as strong winds, low temperatures, and concentrated rainfall. The impact will cover the entire base area."
"What about biology?" Chen Hao asked.
"Based on infrared scanning and footprint analysis, there are signs of large-scale activity in the northern forest area. The animals weigh over 300 kilograms and moved at a slow speed, but their numbers are unknown. It cannot be ruled out that they are a migrating population."
"Three hundred kilograms?" Chen Hao's eyes widened. "That thing would crack if it hit a wall."
“So we have to reinforce the exterior walls,” Susan said, “especially the living area side. Also, the water storage system must be sealed, otherwise a heavy rain could wash silt into the filtration tank.”
Carl made two marks on the paper: "I also suggest increasing the frequency of nighttime inspections. Even just a quick walk around can uncover problems."
"Wait a minute." Chen Hao raised his hand. "One of you is talking about preventing natural disasters, another is talking about preventing monsters, and the third is staring at the computer, afraid it will develop self-awareness... Have we forgotten the main point?"
"What's the key point?" Susan asked.
“There are only four of us,” Chen Hao said. “Even the most capable people can’t be in two places at once. Today we’re fixing welding torches, tomorrow we’re checking logs, and the day after tomorrow we have to carry sandbags to block the door. Who can keep doing this forever?”
The room was silent for a second.
"Are you tired of talking?" Susan asked, looking at him.
"I'm not tired," Chen Hao said, leaning against the wall, his voice lowering. "I want to ask, what are we striving for all this time? Just to live in a house that doesn't leak, and then live in constant fear of the next wave of trouble?"
No one spoke.
Susan glanced down at her notebook and turned a page: "We're repairing walls, getting electricity, and building a kitchen, not to prepare for tomorrow, but to prevent tomorrow from crushing us."
"But what if the rain stops tomorrow?" Chen Hao asked. "What if the rain just keeps pouring down, the machines go crazy, and there are big guys banging on the door outside? Are the four of us going to stand at the door and swing hammers at them?"
“Then it’s time for the hammer to break.” Susan looked up. “But we have to stand there.”
Carl put down his pen: "True survival is not about waiting for disaster to pass, but about preventing disaster from harming you in advance."
"So you plan to keep preparing indefinitely?"
"Otherwise?" Karl retorted, "Wait until everything hits you in the face before you start crying?"
Nana looked at the two of them and finally spoke: "The system is learning, and we must evolve too. It can switch lines on its own, so we can learn to react faster. It can predict load, so we can predict storms. The only difference is—it's a program, and we're human."
Chen Hao stood still, without moving.
After a few seconds, he suddenly laughed: "Okay, you all talk like it's the truth."
He walked to the middle, clapped his hands twice, and said, "Then what are we waiting for? Starting tomorrow, those who need to stockpile grain should stockpile grain, those who need to practice should practice, and those who need to keep a close eye on the system should keep a close eye on it. Nobody is going to shirk their responsibilities!"
Susan closed her notebook: "I'll be in charge of taking inventory and making a list of emergency supplies."
“I adjusted the circuit layout,” Carl said. “I added isolation gates to prevent the system from overstepping its authority.”
“I monitor external signals and weather data,” Nana said. “I issue an immediate warning if any abnormal fluctuations are detected.”
Chen Hao thought for a moment and said, "Then I... will be in charge of moving things."
“You’re in charge of overall coordination,” Susan corrected. “You’ll arrange who does what. Don’t end up claiming heatstroke after carrying only half a bag of rice.”
"That's different. Last time the sun was really too strong."
"This time, the brim of your hat will press down on your eyebrows."
Carl scribbled a few lines on a piece of paper, a rough outline of tomorrow's tasks. Nana continued checking her log, but her pace slowed. Susan stared at the overhead light, its glow illuminating her face; her expression was blank, but her gaze was steady.
Chen Hao sat on the ground, twirling a welding rod in his hand.
“Actually,” he suddenly said, “I never studied for exams before. When the teacher said ‘there’s a quiz next week,’ I would just ignore it. As a result, I always failed and had to retake the exam.”
"And then?" Susan asked.
"Then I thought, if I had just glanced at the books earlier, I wouldn't have failed the course." He dangled his welding rod. "Now that I think about it, it's the same with life. If you don't prepare in advance, you'll just get beaten up when trouble comes."
"So you're willing to move now?" Karl looked up.
“It’s not that we wanted to,” Chen Hao grinned. “It’s that we realized something—the four of us are here not just by chance, but we have to survive together.”
The lights in the room were still on.
The projector was off, the blueprints lay spread out on the floor, and the monitor screen flickered with a faint standby light. None of the four people left, nor did they say the meeting was over; it was as if something had settled after that last sentence.
Chen Hao stuck the welding rod into the ground: "We'll start work at six o'clock tomorrow morning. Anyone who's late will have to clean the toilets for a week."
"You set the time?" Carl raised an eyebrow.
"I propose it," Chen Hao said with a smile, "you can object."
No one objected.
Susan quietly put the notebook into her bag. Nana placed the tablet on her lap, swiped her finger to bring up a new monitoring interface. Carl added an extra column to the paper: Emergency Response Procedures.
Chen Hao leaned against the wall, closed his eyes for a while, and then opened them again.
“I said,” he looked at Nana, “if one day the system really thinks it can be in charge, could we show it a video called ‘The Consequences of a Robot Rebellion’?”
“I can download it,” Nana said. “And add subtitles: None of the main characters have a good ending.”
"That's right." Chen Hao nodded. "Let it know that robots that cause trouble never get away with it."
As soon as he finished speaking, the light above his head flickered once.
It was very short, just a moment.
All four looked up at the same time.
The light returned to normal.
But Nana had already opened the tablet and quickly tapped into the list of background processes.
Continue read on readnovelmtl.com