Chapter 309 Model Collapse (4506)
Even after the last red ripple disappeared, Fang Yu was still worried. He searched his brain carefully three times again to confirm that there were no red ripples of Vernis remaining in his brain. Only then did he breathe a sigh of relief.
This extremely detailed inner vision is very energy-consuming. Even though Fang Yu has strengthened his physical strength to the peak level of carbon-based creatures, a few drops of sweat still seep out of his forehead.
Phew, that was close.
Wiping the sweat off his face, Fang Yu couldn't help but feel scared.
Fortunately, I obtained the "God Model" from Aizen Myōo, fortunately, I discovered the changes in my feelings towards Chen Shuyao in time, and fortunately, I was self-reflective enough to discover the changes in my self-cognition.
How could I call myself the God of Desire?
How could I have identified with the losers who had abandoned their physical bodies and escaped into the astral realm? How could I have been so smug about being the God of Desire?
Would Sun Wukong be proud of his title of Fighting Buddha? He would only feel ashamed!
As for myself, I twice claimed to be a god in Miami, using the word "God," which has a specific connotation in the West, and which may have contained some of my own bad taste.
But just now, I subconsciously identified myself as the God of Desire in my heart, which is very abnormal.
If it were just these, it would be fine. Maybe it was just my vanity that made me want to play tricks on others.
What really made Fang Yu realize that something was wrong was his condescending and indifferent attitude when he saw Chen Shuyao dressing up carefully to meet Musk.
It has been a while since I entered the arcane world. Although I felt a sense of superiority over ordinary people, I never treated the people around me as straw dogs.
However, just now, I subconsciously developed a mentality of looking down on Chen Shuyao, like an ant, "As long as I want, you are just an XX."
This is definitely not normal.
The great arcanists of the Great Arcane Age didn't become so arrogant, so how could I possibly become so arrogant?
If I were this kind of person, I would have been given a title like "The Four Young Masters of Haixi" and everyone would know me, right?
When did you start to change?
Thinking about it carefully, it seems that since the time when Vernes invaded the sea of consciousness, my mentality has undergone subtle changes.
This change was different from the emotional change caused by being immersed in the belief of desire at the Four Seasons Hotel in Los Santos two months ago. Instead, it was a subtle and unconscious change in one's perception and attitude towards certain things.
That's why I have been doing things more and more unscrupulously these days.
After a while, when Vernis's model completely invades his brain, even if he is not possessed by this despicable god, he will definitely become a complete evil god!
That was a close call.
etc.
So, during this time, I've been with so many models and female celebrities...
Well, the means used and the attitude afterwards may have something to do with Vernis’s model parasitism, but the act of “doing” this thing should still be out of one’s own will.
After all, Vernis's parasitic model has now completely disappeared, and I don't think I made a mistake in the past two months.
Looking back now, the newly added Victoria Garrity and Camila Moreau taste quite good.
Fang Yu did another full-body scan and was sure that the model left by Vernis had indeed been reduced to ashes.
"Knock knock knock." There was a knock on the door.
"Fang Yu, Fang Yu, are you okay?"
"Cousin, are you sick?"
"Don't you have classes this afternoon? Why are you here? Do you have any professionalism when addressing people by their titles or names?"
"What's it to you? Humph, anyway, my brother and I don't plan to work after we get married, so I'll just stay at home and take care of the kids."
"I don't plan on having a baby so early and asking you to take care of it. If you want to be a nanny, I have your resume and will upload it to 58.com for you in a moment."
…
Hearing Lu Jiayan and Shen Shuyao quarreling in low voices outside the door, Fang Yu suddenly felt as if he was in another world.
It was as if everything I had experienced in the past few days was covered by a veil, and now the veil was suddenly lifted.
"I'm fine, I just checked backstage."
Fang Yu opened the door of his private laboratory, one in his left hand and one in his right hand, feeling very happy.
"Are you really okay?" Chen Shuyao touched Fang Yu's forehead worriedly, and then carefully looked at Fang Yu's complexion. Sure enough, there was nothing wrong. Then she continued, "Du Changgeng seems to have found something. Do you want to go over and take a look?"
Oh, by the way, there is more important business to do.
Fang Yu slapped his forehead, kissed both of them on the cheek, and quickly walked to the open office area.
"Boss! We did ten rounds of testing, and the results..." Team leaders Du Changgeng, Hua Zecheng, Yang Zilong, and Lu Si all had serious expressions on their faces, and it was obvious that the situation was not very optimistic.
Someone pushed a chair towards Fang Yu. Fang Yu sat down and looked at the monitor, and couldn't help frowning.
After ten rounds of training, Model 2, using the original data, showed no performance degradation. Overfitting can be corrected through regularization and data cleaning.
"Model 1, which was trained using images generated by the Orange model, showed obvious overfitting. Boss, take a look! After ten rounds, the generated images began to converge."
“And Model 3, which is trained using data generated by Model 1 in each round…”
A trace of worry crossed Du Changgeng's face. "By the fifth round, Model 3's overfitting had already surpassed that of Model 1, which had undergone ten rounds of training. By the ninth round, the entire model was on the verge of collapse and could barely produce meaningful data."
"We were surprised by this result, so we used data generated by different models to train a new 3B sub-model, named Model 4. The final performance after training was very similar to that of Model 1, with no improvement."
"Initially, we thought it was a data quality issue. There might be some distortion in the image data of the Orange model, so we repeated the training using text."
"The error rate of the text data generated by the Orange Big model was much lower than that of the original data used by Model 2. However, Model 1, which was ultimately trained using the text data generated by the Orange Big model, still experienced performance degradation. After multiple rounds of training, the overfitting phenomenon was nearly five times higher than that of Model 2."
"And in this case, the Model 3 also experienced a model breakdown after eight or nine rounds."
"In other words, regardless of the quality of AI-generated data, using it to train a large model will cause the performance of the large model to degrade. This... this is simply incredible!"
"It's almost like..."
"Inbreeding?" Lu Jiayan asked curiously.
"Yes! Inbreeding! That's right!" Du Changgeng clapped his hands and looked at Lu Jiayan in surprise. He didn't expect that this young proprietress, who was obviously an outsider, had such a good understanding.
Fang Yu touched his chin thoughtfully.
He had already guessed this result, otherwise he would not have proposed conducting a comparative experiment.
The problem discovered by Yang Zilong and Du Changgeng does seem a bit strange at first glance. The same data, processed by AI and then used to train another AI, is actually not as effective as the manually cleaned data of lower quality.
Using the data generated by this trained AI to continue training AI, no matter which AI is being trained, will cause performance to further decline.
This question may seem strange, but in fact, with a little thought, you can understand that it is probably a problem caused by the continuous simplification of data diversity and complexity by AI.
Not to mention the orange model, even for grapefruit, the data generated cannot be 100% correct. Although the errors may be less than those in artificial data, the errors lack diversity and complexity.
To put it bluntly, AI is smart in all ways, but humans are stupid in all kinds of ways.
But from the perspective of racial inheritance, stupid and weird humans have stronger racial inheritance capabilities than AI that they create and is smarter than humans.
Wow? Come to think of it, a fool like Meng Guangkang has made a significant contribution to the inheritance of human civilization?
Almost cured my stupidity?
By repeatedly using this type of data for training, similar errors will be considered real by the large model, thus becoming a "stamp of thought" and continuing to produce more erroneous content.
Using these contents for training again will produce a "bias-variance trade-off", that is, the content with higher probability will be further strengthened, and the content with lower probability will be continuously ignored.
The next generation of models will actually result in more data loss.
At the same time, due to the characteristics of large models, "function approximation errors" will occur. Large models will retain the wrong things they have learned. After accumulation from generation to generation, they will eventually become all errors, thus losing the ability to correct errors.
This situation is indeed very similar to inbreeding depression caused by inbreeding.
From this perspective, large AI models actually already possess certain biological properties.
It's just that the population of this species is too homogeneous, resulting in inbreeding.
This problem is not too urgent at present. After all, there is a lot of raw data available on the Internet. The problem can be solved by increasing investment in data sets, obtaining more raw data, and increasing the manpower and methods of data preprocessing.
But in the long run—no, not too long—the explosion of AI will be exponential. Perhaps in two to three years, over 70% of the data on the internet will be AI-generated, and humans will be unable to distinguish whether it's AI-generated data or real, human-generated data.
At that time, no matter how large the parameters of the AI model are, it may face performance degradation or collapse due to the contamination of training data by AI-generated data.
"Thirty trillion." Fang Yu quickly estimated a number.
???
Several team leaders all had question marks on their faces.
"The available valid data on the internet can probably support the Orange model up to a scale of 30 trillion parameters. After that, it will inevitably be affected by inbreeding."
Fang Yu pointed to the data on the monitor.
Thirty trillion is not so bad. The Orange model currently being trained has only 1.5 trillion parameters, but it has already begun to have the initial prototype of strong artificial intelligence.
The 30 trillion-scale model will surely have achieved super artificial intelligence. By then, the problem of diversity in AI-generated data should have been solved.
Then there's no problem.
Several team leaders breathed a sigh of relief.
Is everything alright?
Fang Yu's eyes flashed.
Even if the Orange model achieves super artificial intelligence, this problem will still exist as long as the diversity issue is not resolved.
Even if various technical means are used to increase the complexity of AI data as much as possible and the model does not crash in the end, this large model is destined to be unable to carry its own vision.
Some of the ideas I came up with from referring to the "God Model" can actually solve this problem...
However, before that, we still have to take over the search engine business first.
Only by taking over the search engine can we have enough data resources to support the evolution of the Orange model into a super artificial intelligence.
Only after the Orange model evolves into super artificial intelligence, the second-generation model generated with reference to the "God Model" can be truly useful.
"Shu Yao, notify the 'Y' project team. I want to hear the progress report of the 'Y' project in five minutes."
Fang Yu looked at the time. It was already four o'clock in the afternoon. In another half hour, the relevant leaders of Haixi County should bring Musk to the company, which would be just enough time for a meeting.
I have to go to Xinhao at 11 o'clock in the evening to catch up with Da Mimi and Repa. Chihiro has put a lot of pressure on them recently, so I have to go and help them relieve the pressure.
It just so happens that I can meet my grandfather's introducer in Xinhao tomorrow morning. The person my grandfather introduced should be reliable, but I still have to meet him first.
I have to go to Los Santos at noon tomorrow. Scarlett went to Middle-earth to shoot Ghost in the Shell. Kendall Jenner was quite reserved and just harassed her via iMessage. Her bestie Gigi Hadid, on the other hand, kept sending provocative messages on Snapchat.
I must teach him a lesson!
"This 'fencing gate' scandal is a blow to both factions, but it's a bigger blow to Trump. Karl Rove even suggested that the congratulatory faction should convene a national committee to disqualify Trump from running for office."
While Musk was on the phone, he was also sending emails to various SpaceX employees. On the other end of the phone, his chief of staff, Sam Taylor, was telling him about the current situation in Miami.
For a businessman of his level, whether he hates Zheng Zhi or not, he must care about Zheng Zhi.
"It's difficult. At least two-thirds of the committee members need to vote in favor, which is almost impossible. It looks like Hillary will win this time." Musk's fingers tapped quickly on the computer.
"'Fencing Gate' broke out too late. The Indiana primary scandal had already concluded counting votes the day after. Although the national convention had not yet been held to secure the party nomination, Trump had already secured the nomination votes in advance."
"If Trump is forced to withdraw from the race at this point, the congratulatory faction will likely cause a split in the party. Even if Paul Ryan and Mitch McConnell don't support Trump, they won't risk a split in the party by pushing for a replacement nominee. Of course, the prerequisite is that the Trump family immediately cuts ties with Jared Kushner. But if that happens, the Jude Foundation will undoubtedly fully support Hillary Clinton."
"You're right, Elon. It's widely speculated that Ivanka might announce her divorce from Jared Kushner in the next few days, distancing herself from the family and playing the victim. However, in that video, Jared said the Trump family was just nouveau riche, and that Ivanka married him just to gain support from Jude's consortium. This will still be a huge blow to Trump."
"Boss, we've already started landing. We'll be below 3,000 meters in five minutes, and the Wi-Fi on the plane will be shut down."
Musk's captain informed Musk using a loud speaker in the cabin.
"You heard me, Sam, I'm going to disconnect first, stay connected."
"Okay, Elon, it's the same if I go this time. There's no need for you to go there yourself."
"No, this time I have to see this Orange Technology in person." Elon Musk shook his head. "At least 10,000 AI applications have been created in the past month. 50% of them use the Orange Big Model API. The rest also use the open-source Orange Big Model, and this number continues to rise."
"I'm confident that the 300 billion parameter Juzi2.5 that's already in operation isn't the full package. They probably already have a large model with a trillion parameters or even more."
"I have to see with my own eyes what kind of progress they have made."
Musk ended the call and looked down through the window.
The clouds were so thick that I couldn't see anything.
(End of this chapter)
Continue read on readnovelmtl.com
