Artificial General Intelligence

Kommentarer · 49 Visninger

Artificial general intelligence (AGI) is a kind of artificial intelligence (AI) that matches or exceeds human cognitive abilities across a large range of cognitive jobs.

Artificial basic intelligence (AGI) is a kind of artificial intelligence (AI) that matches or exceeds human cognitive abilities across a vast array of cognitive jobs. This contrasts with narrow AI, which is restricted to specific tasks. [1] Artificial superintelligence (ASI), on the other hand, describes AGI that greatly goes beyond human cognitive abilities. AGI is considered among the definitions of strong AI.


Creating AGI is a primary goal of AI research and of business such as OpenAI [2] and Meta. [3] A 2020 study determined 72 active AGI research and advancement tasks throughout 37 countries. [4]

The timeline for achieving AGI remains a topic of ongoing dispute amongst researchers and experts. As of 2023, some argue that it may be possible in years or years; others keep it may take a century or longer; a minority believe it may never be accomplished; and another minority declares that it is already here. [5] [6] Notable AI researcher Geoffrey Hinton has actually revealed concerns about the quick progress towards AGI, recommending it could be attained earlier than numerous expect. [7]

There is debate on the specific meaning of AGI and regarding whether modern big language designs (LLMs) such as GPT-4 are early types of AGI. [8] AGI is a typical topic in sci-fi and futures research studies. [9] [10]

Contention exists over whether AGI represents an existential danger. [11] [12] [13] Many professionals on AI have stated that reducing the risk of human termination posed by AGI should be a global concern. [14] [15] Others discover the development of AGI to be too remote to present such a threat. [16] [17]

Terminology


AGI is also referred to as strong AI, [18] [19] full AI, [20] human-level AI, [5] human-level intelligent AI, or general intelligent action. [21]

Some academic sources schedule the term "strong AI" for computer system programs that experience life or awareness. [a] On the other hand, weak AI (or narrow AI) has the ability to resolve one specific problem but does not have general cognitive capabilities. [22] [19] Some academic sources utilize "weak AI" to refer more broadly to any programs that neither experience consciousness nor have a mind in the very same sense as human beings. [a]

Related ideas consist of artificial superintelligence and transformative AI. An artificial superintelligence (ASI) is a hypothetical type of AGI that is a lot more typically smart than humans, [23] while the notion of transformative AI connects to AI having a big influence on society, for example, similar to the agricultural or commercial transformation. [24]

A structure for categorizing AGI in levels was proposed in 2023 by Google DeepMind researchers. They specify 5 levels of AGI: emerging, qualified, expert, virtuoso, and superhuman. For example, a competent AGI is defined as an AI that outperforms 50% of proficient adults in a broad variety of non-physical jobs, and a superhuman AGI (i.e. a synthetic superintelligence) is likewise defined however with a threshold of 100%. They consider large language models like ChatGPT or LLaMA 2 to be instances of emerging AGI. [25]

Characteristics


Various popular meanings of intelligence have actually been proposed. One of the leading proposals is the Turing test. However, there are other popular definitions, and some researchers disagree with the more popular approaches. [b]

Intelligence qualities


Researchers normally hold that intelligence is needed to do all of the following: [27]

factor, wiki.fablabbcn.org usage technique, solve puzzles, and make judgments under uncertainty
represent knowledge, including good sense knowledge
strategy
discover
- interact in natural language
- if necessary, integrate these abilities in conclusion of any offered goal


Many interdisciplinary approaches (e.g. cognitive science, computational intelligence, and choice making) consider additional traits such as creativity (the capability to form novel psychological images and ideas) [28] and autonomy. [29]

Computer-based systems that display a lot of these abilities exist (e.g. see computational creativity, automated thinking, decision support system, robotic, evolutionary computation, smart representative). There is dispute about whether modern-day AI systems possess them to an appropriate degree.


Physical characteristics


Other capabilities are considered preferable in intelligent systems, as they may impact intelligence or help in its expression. These consist of: [30]

- the capability to sense (e.g. see, hear, and so on), and
- the ability to act (e.g. relocation and control things, change location to explore, and so on).


This consists of the ability to detect and react to danger. [31]

Although the capability to sense (e.g. see, hear, etc) and the ability to act (e.g. relocation and control items, modification place to check out, etc) can be desirable for some intelligent systems, [30] these physical capabilities are not strictly needed for an entity to certify as AGI-particularly under the thesis that large language designs (LLMs) may already be or become AGI. Even from a less positive viewpoint on LLMs, there is no firm requirement for an AGI to have a human-like type; being a silicon-based computational system suffices, provided it can process input (language) from the external world in location of human senses. This interpretation lines up with the understanding that AGI has actually never been proscribed a particular physical personification and hence does not demand a capability for mobility or traditional "eyes and ears". [32]

Tests for human-level AGI


Several tests indicated to verify human-level AGI have been thought about, consisting of: [33] [34]

The concept of the test is that the maker has to try and pretend to be a guy, by answering concerns put to it, and it will only pass if the pretence is fairly convincing. A considerable portion of a jury, who must not be expert about makers, should be taken in by the pretence. [37]

AI-complete problems


An issue is informally called "AI-complete" or "AI-hard" if it is thought that in order to fix it, one would require to carry out AGI, due to the fact that the option is beyond the abilities of a purpose-specific algorithm. [47]

There are numerous problems that have actually been conjectured to require general intelligence to fix as well as humans. Examples consist of computer vision, natural language understanding, and handling unexpected scenarios while solving any real-world problem. [48] Even a specific task like translation needs a maker to check out and write in both languages, follow the author's argument (factor), understand the context (knowledge), and consistently recreate the author's initial intent (social intelligence). All of these issues require to be solved simultaneously in order to reach human-level device efficiency.


However, many of these tasks can now be performed by modern-day large language models. According to Stanford University's 2024 AI index, AI has actually reached human-level performance on many benchmarks for checking out comprehension and visual thinking. [49]

History


Classical AI


Modern AI research study began in the mid-1950s. [50] The very first generation of AI researchers were convinced that artificial basic intelligence was possible which it would exist in just a couple of decades. [51] AI leader Herbert A. Simon composed in 1965: "makers will be capable, within twenty years, of doing any work a male can do." [52]

Their forecasts were the motivation for Stanley Kubrick and Arthur C. Clarke's character HAL 9000, who embodied what AI researchers thought they might create by the year 2001. AI pioneer Marvin Minsky was an expert [53] on the project of making HAL 9000 as realistic as possible according to the agreement forecasts of the time. He stated in 1967, "Within a generation ... the problem of producing 'artificial intelligence' will significantly be solved". [54]

Several classical AI projects, such as Doug Lenat's Cyc job (that began in 1984), and Allen Newell's Soar task, were directed at AGI.


However, in the early 1970s, it ended up being obvious that researchers had grossly underestimated the problem of the job. Funding firms became skeptical of AGI and put scientists under increasing pressure to produce helpful "applied AI". [c] In the early 1980s, Japan's Fifth Generation Computer Project restored interest in AGI, setting out a ten-year timeline that included AGI goals like "carry on a table talk". [58] In response to this and the success of professional systems, both market and government pumped money into the field. [56] [59] However, self-confidence in AI amazingly collapsed in the late 1980s, and the objectives of the Fifth Generation Computer Project were never fulfilled. [60] For the 2nd time in 20 years, AI scientists who anticipated the impending achievement of AGI had been mistaken. By the 1990s, AI researchers had a credibility for making vain promises. They became hesitant to make forecasts at all [d] and avoided reference of "human level" expert system for worry of being labeled "wild-eyed dreamer [s]. [62]

Narrow AI research


In the 1990s and early 21st century, mainstream AI attained business success and academic respectability by focusing on specific sub-problems where AI can produce verifiable results and business applications, such as speech recognition and recommendation algorithms. [63] These "applied AI" systems are now used thoroughly throughout the innovation industry, and research study in this vein is greatly moneyed in both academia and industry. Since 2018 [update], development in this field was considered an emerging pattern, and a mature stage was expected to be reached in more than 10 years. [64]

At the turn of the century, many traditional AI scientists [65] hoped that strong AI might be developed by combining programs that resolve numerous sub-problems. Hans Moravec composed in 1988:


I am confident that this bottom-up route to expert system will one day meet the standard top-down path majority way, prepared to offer the real-world skills and the commonsense knowledge that has actually been so frustratingly evasive in thinking programs. Fully intelligent devices will result when the metaphorical golden spike is driven unifying the 2 efforts. [65]

However, even at the time, this was contested. For example, Stevan Harnad of Princeton University concluded his 1990 paper on the symbol grounding hypothesis by mentioning:


The expectation has actually often been voiced that "top-down" (symbolic) approaches to modeling cognition will somehow fulfill "bottom-up" (sensory) approaches somewhere in between. If the grounding considerations in this paper stand, then this expectation is hopelessly modular and there is truly just one viable path from sense to signs: from the ground up. A free-floating symbolic level like the software application level of a computer will never ever be reached by this route (or vice versa) - nor is it clear why we need to even try to reach such a level, because it appears getting there would simply total up to uprooting our signs from their intrinsic meanings (thereby merely decreasing ourselves to the functional equivalent of a programmable computer). [66]

Modern artificial general intelligence research


The term "synthetic general intelligence" was used as early as 1997, by Mark Gubrud [67] in a discussion of the implications of fully automated military production and operations. A mathematical formalism of AGI was proposed by Marcus Hutter in 2000. Named AIXI, the proposed AGI representative increases "the capability to satisfy objectives in a broad range of environments". [68] This type of AGI, identified by the ability to increase a mathematical meaning of intelligence rather than show human-like behaviour, [69] was also called universal expert system. [70]

The term AGI was re-introduced and promoted by Shane Legg and Ben Goertzel around 2002. [71] AGI research study activity in 2006 was explained by Pei Wang and Ben Goertzel [72] as "producing publications and preliminary outcomes". The first summertime school in AGI was organized in Xiamen, China in 2009 [73] by the Xiamen university's Artificial Brain Laboratory and OpenCog. The first university course was given up 2010 [74] and 2011 [75] at Plovdiv University, Bulgaria by Todor Arnaudov. MIT presented a course on AGI in 2018, arranged by Lex Fridman and including a number of guest lecturers.


Since 2023 [update], a small number of computer system scientists are active in AGI research, and lots of add to a series of AGI conferences. However, significantly more scientists have an interest in open-ended learning, [76] [77] which is the idea of enabling AI to constantly learn and innovate like humans do.


Feasibility


As of 2023, the advancement and potential achievement of AGI remains a topic of extreme argument within the AI community. While standard agreement held that AGI was a remote objective, current improvements have actually led some researchers and industry figures to declare that early forms of AGI might already exist. [78] AI pioneer Herbert A. Simon speculated in 1965 that "devices will be capable, within twenty years, of doing any work a man can do". This forecast stopped working to come real. Microsoft co-founder Paul Allen thought that such intelligence is not likely in the 21st century due to the fact that it would require "unforeseeable and essentially unpredictable advancements" and a "clinically deep understanding of cognition". [79] Writing in The Guardian, roboticist Alan Winfield declared the gulf in between modern-day computing and human-level expert system is as wide as the gulf in between current space flight and practical faster-than-light spaceflight. [80]

An additional challenge is the absence of clearness in defining what intelligence involves. Does it need consciousness? Must it display the ability to set objectives along with pursue them? Is it simply a matter of scale such that if design sizes increase sufficiently, intelligence will emerge? Are centers such as planning, thinking, and causal understanding needed? Does intelligence require clearly reproducing the brain and its specific faculties? Does it need feelings? [81]

Most AI researchers think strong AI can be attained in the future, however some thinkers, like Hubert Dreyfus and Roger Penrose, deny the possibility of achieving strong AI. [82] [83] John McCarthy is amongst those who believe human-level AI will be accomplished, but that today level of progress is such that a date can not accurately be forecasted. [84] AI specialists' views on the feasibility of AGI wax and subside. Four polls performed in 2012 and 2013 recommended that the median price quote among professionals for when they would be 50% positive AGI would show up was 2040 to 2050, depending on the survey, with the mean being 2081. Of the professionals, 16.5% answered with "never" when asked the exact same question but with a 90% self-confidence rather. [85] [86] Further existing AGI development factors to consider can be found above Tests for verifying human-level AGI.


A report by Stuart Armstrong and Kaj Sotala of the Machine Intelligence Research Institute found that "over [a] 60-year timespan there is a strong predisposition towards forecasting the arrival of human-level AI as between 15 and 25 years from the time the prediction was made". They analyzed 95 forecasts made between 1950 and 2012 on when human-level AI will happen. [87]

In 2023, Microsoft researchers released a comprehensive examination of GPT-4. They concluded: "Given the breadth and depth of GPT-4's capabilities, our company believe that it could reasonably be viewed as an early (yet still insufficient) version of an artificial basic intelligence (AGI) system." [88] Another study in 2023 reported that GPT-4 outperforms 99% of people on the Torrance tests of creativity. [89] [90]

Blaise Agüera y Arcas and Peter Norvig wrote in 2023 that a considerable level of general intelligence has actually already been attained with frontier designs. They composed that unwillingness to this view originates from four primary factors: a "healthy skepticism about metrics for AGI", an "ideological dedication to alternative AI theories or strategies", a "devotion to human (or biological) exceptionalism", or a "issue about the financial implications of AGI". [91]

2023 likewise marked the development of large multimodal designs (large language designs efficient in processing or producing multiple methods such as text, audio, and images). [92]

In 2024, OpenAI released o1-preview, the first of a series of models that "invest more time believing before they react". According to Mira Murati, this ability to believe before responding represents a new, additional paradigm. It enhances model outputs by investing more computing power when creating the response, whereas the model scaling paradigm improves outputs by increasing the design size, training data and training calculate power. [93] [94]

An OpenAI worker, Vahid Kazemi, declared in 2024 that the company had actually accomplished AGI, stating, "In my viewpoint, we have actually already accomplished AGI and it's even more clear with O1." Kazemi clarified that while the AI is not yet "much better than any human at any job", it is "better than a lot of humans at many jobs." He likewise resolved criticisms that large language designs (LLMs) simply follow predefined patterns, comparing their learning procedure to the scientific technique of observing, hypothesizing, and confirming. These declarations have actually stimulated debate, as they rely on a broad and non-traditional definition of AGI-traditionally understood as AI that matches human intelligence across all domains. Critics argue that, while OpenAI's models demonstrate remarkable versatility, they may not totally fulfill this requirement. Notably, Kazemi's comments came shortly after OpenAI eliminated "AGI" from the terms of its collaboration with Microsoft, prompting speculation about the company's tactical objectives. [95]

Timescales


Progress in artificial intelligence has actually traditionally gone through periods of fast development separated by periods when development appeared to stop. [82] Ending each hiatus were fundamental advances in hardware, software or both to develop space for further progress. [82] [98] [99] For instance, the computer hardware available in the twentieth century was not sufficient to implement deep knowing, which requires big numbers of GPU-enabled CPUs. [100]

In the introduction to his 2006 book, [101] Goertzel states that quotes of the time required before a really flexible AGI is constructed vary from 10 years to over a century. As of 2007 [update], the agreement in the AGI research study community appeared to be that the timeline talked about by Ray Kurzweil in 2005 in The Singularity is Near [102] (i.e. in between 2015 and 2045) was possible. [103] Mainstream AI researchers have given a wide variety of opinions on whether development will be this fast. A 2012 meta-analysis of 95 such opinions found a bias towards anticipating that the onset of AGI would happen within 16-26 years for modern-day and historical forecasts alike. That paper has actually been slammed for how it classified viewpoints as specialist or non-expert. [104]

In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton established a neural network called AlexNet, which won the ImageNet competitors with a top-5 test error rate of 15.3%, significantly much better than the second-best entry's rate of 26.3% (the traditional technique utilized a weighted amount of scores from various pre-defined classifiers). [105] AlexNet was concerned as the preliminary ground-breaker of the current deep knowing wave. [105]

In 2017, researchers Feng Liu, Yong Shi, and Ying Liu performed intelligence tests on openly available and freely accessible weak AI such as Google AI, Apple's Siri, and others. At the maximum, these AIs reached an IQ value of about 47, which corresponds approximately to a six-year-old child in first grade. A grownup comes to about 100 on average. Similar tests were carried out in 2014, with the IQ rating reaching a maximum value of 27. [106] [107]

In 2020, OpenAI developed GPT-3, a language model capable of performing lots of varied jobs without specific training. According to Gary Grossman in a VentureBeat article, while there is agreement that GPT-3 is not an example of AGI, it is considered by some to be too advanced to be classified as a narrow AI system. [108]

In the exact same year, Jason Rohrer utilized his GPT-3 account to establish a chatbot, and offered a chatbot-developing platform called "Project December". OpenAI requested modifications to the chatbot to abide by their security guidelines; Rohrer disconnected Project December from the GPT-3 API. [109]

In 2022, DeepMind developed Gato, a "general-purpose" system capable of carrying out more than 600 different jobs. [110]

In 2023, Microsoft Research published a research study on an early variation of OpenAI's GPT-4, competing that it displayed more basic intelligence than previous AI models and showed human-level efficiency in tasks spanning multiple domains, such as mathematics, coding, and law. This research study sparked an argument on whether GPT-4 could be thought about an early, insufficient version of artificial basic intelligence, emphasizing the need for additional expedition and examination of such systems. [111]

In 2023, the AI scientist Geoffrey Hinton stated that: [112]

The concept that this stuff might really get smarter than individuals - a few people believed that, [...] But the majority of individuals thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.


In May 2023, Demis Hassabis similarly said that "The development in the last couple of years has actually been quite incredible", and that he sees no reason that it would slow down, expecting AGI within a years or perhaps a couple of years. [113] In March 2024, Nvidia's CEO, Jensen Huang, mentioned his expectation that within 5 years, AI would can passing any test at least in addition to people. [114] In June 2024, the AI scientist Leopold Aschenbrenner, a former OpenAI worker, approximated AGI by 2027 to be "noticeably plausible". [115]

Whole brain emulation


While the advancement of transformer models like in ChatGPT is considered the most promising path to AGI, [116] [117] whole brain emulation can serve as an alternative method. With entire brain simulation, a brain model is developed by scanning and mapping a biological brain in information, and after that copying and mimicing it on a computer system or another computational device. The simulation model should be sufficiently loyal to the original, so that it acts in practically the exact same method as the original brain. [118] Whole brain emulation is a type of brain simulation that is gone over in computational neuroscience and neuroinformatics, and for medical research study functions. It has been discussed in expert system research study [103] as an approach to strong AI. Neuroimaging innovations that could deliver the required in-depth understanding are improving rapidly, and futurist Ray Kurzweil in the book The Singularity Is Near [102] anticipates that a map of adequate quality will end up being offered on a similar timescale to the computing power needed to replicate it.


Early estimates


For low-level brain simulation, a really powerful cluster of computer systems or GPUs would be needed, offered the huge amount of synapses within the human brain. Each of the 1011 (one hundred billion) neurons has on average 7,000 synaptic connections (synapses) to other neurons. The brain of a three-year-old child has about 1015 synapses (1 quadrillion). This number declines with age, stabilizing by their adult years. Estimates differ for an adult, varying from 1014 to 5 × 1014 synapses (100 to 500 trillion). [120] A quote of the brain's processing power, based on an easy switch model for neuron activity, is around 1014 (100 trillion) synaptic updates per second (SUPS). [121]

In 1997, Kurzweil took a look at different estimates for the hardware required to equal the human brain and embraced a figure of 1016 calculations per second (cps). [e] (For comparison, if a "calculation" was comparable to one "floating-point operation" - a step utilized to rate current supercomputers - then 1016 "calculations" would be comparable to 10 petaFLOPS, achieved in 2011, while 1018 was accomplished in 2022.) He utilized this figure to forecast the necessary hardware would be readily available sometime between 2015 and 2025, if the rapid development in computer system power at the time of writing continued.


Current research study


The Human Brain Project, an EU-funded effort active from 2013 to 2023, has actually established a particularly in-depth and openly accessible atlas of the human brain. [124] In 2023, scientists from Duke University performed a high-resolution scan of a mouse brain.


Criticisms of simulation-based methods


The synthetic neuron design assumed by Kurzweil and used in many existing synthetic neural network applications is easy compared with biological nerve cells. A brain simulation would likely have to capture the in-depth cellular behaviour of biological nerve cells, currently comprehended only in broad summary. The overhead introduced by complete modeling of the biological, chemical, and physical information of neural behaviour (particularly on a molecular scale) would need computational powers a number of orders of magnitude larger than Kurzweil's price quote. In addition, the estimates do not account for glial cells, which are understood to contribute in cognitive processes. [125]

An essential criticism of the simulated brain approach stems from embodied cognition theory which asserts that human embodiment is a vital aspect of human intelligence and is needed to ground meaning. [126] [127] If this theory is right, any fully functional brain model will need to encompass more than just the nerve cells (e.g., a robotic body). Goertzel [103] proposes virtual personification (like in metaverses like Second Life) as a choice, but it is unknown whether this would suffice.


Philosophical viewpoint


"Strong AI" as defined in philosophy


In 1980, philosopher John Searle coined the term "strong AI" as part of his Chinese room argument. [128] He proposed a distinction in between 2 hypotheses about synthetic intelligence: [f]

Strong AI hypothesis: An expert system system can have "a mind" and "awareness".
Weak AI hypothesis: An expert system system can (only) imitate it believes and has a mind and consciousness.


The first one he called "strong" due to the fact that it makes a stronger declaration: it presumes something special has actually happened to the maker that exceeds those capabilities that we can evaluate. The behaviour of a "weak AI" machine would be precisely similar to a "strong AI" device, however the latter would also have subjective conscious experience. This usage is also common in academic AI research study and books. [129]

In contrast to Searle and traditional AI, some futurists such as Ray Kurzweil utilize the term "strong AI" to indicate "human level synthetic general intelligence". [102] This is not the very same as Searle's strong AI, unless it is presumed that consciousness is necessary for human-level AGI. Academic thinkers such as Searle do not believe that is the case, and to most synthetic intelligence researchers the concern is out-of-scope. [130]

Mainstream AI is most interested in how a program behaves. [131] According to Russell and Norvig, "as long as the program works, they don't care if you call it real or a simulation." [130] If the program can act as if it has a mind, then there is no need to understand if it actually has mind - indeed, there would be no chance to inform. For AI research, Searle's "weak AI hypothesis" is equivalent to the statement "synthetic general intelligence is possible". Thus, according to Russell and Norvig, "most AI researchers take the weak AI hypothesis for given, and don't care about the strong AI hypothesis." [130] Thus, for scholastic AI research study, "Strong AI" and "AGI" are two different things.


Consciousness


Consciousness can have numerous meanings, and some aspects play substantial roles in sci-fi and the ethics of expert system:


Sentience (or "sensational awareness"): The capability to "feel" understandings or emotions subjectively, rather than the ability to reason about understandings. Some philosophers, such as David Chalmers, use the term "consciousness" to refer solely to extraordinary awareness, which is roughly comparable to sentience. [132] Determining why and how subjective experience occurs is known as the difficult problem of awareness. [133] Thomas Nagel discussed in 1974 that it "feels like" something to be conscious. If we are not mindful, then it doesn't seem like anything. Nagel uses the example of a bat: we can sensibly ask "what does it feel like to be a bat?" However, we are unlikely to ask "what does it seem like to be a toaster?" Nagel concludes that a bat seems conscious (i.e., has awareness) however a toaster does not. [134] In 2022, a Google engineer declared that the business's AI chatbot, LaMDA, had actually achieved sentience, though this claim was extensively challenged by other professionals. [135]

Self-awareness: To have conscious awareness of oneself as a separate person, specifically to be consciously mindful of one's own ideas. This is opposed to just being the "topic of one's believed"-an os or debugger has the ability to be "knowledgeable about itself" (that is, to represent itself in the exact same way it represents everything else)-but this is not what individuals usually imply when they utilize the term "self-awareness". [g]

These traits have a moral dimension. AI sentience would generate concerns of welfare and legal defense, likewise to animals. [136] Other aspects of awareness associated to cognitive abilities are likewise relevant to the concept of AI rights. [137] Figuring out how to incorporate innovative AI with existing legal and social structures is an emergent problem. [138]

Benefits


AGI could have a large range of applications. If oriented towards such objectives, AGI might help reduce different problems on the planet such as cravings, hardship and health issue. [139]

AGI could improve performance and effectiveness in many jobs. For example, in public health, AGI could accelerate medical research study, significantly against cancer. [140] It could look after the elderly, [141] and democratize access to fast, top quality medical diagnostics. It might offer fun, cheap and personalized education. [141] The requirement to work to subsist could become outdated if the wealth produced is properly rearranged. [141] [142] This likewise raises the concern of the place of human beings in a radically automated society.


AGI could also help to make logical decisions, and to expect and avoid catastrophes. It could likewise help to reap the advantages of potentially devastating technologies such as nanotechnology or climate engineering, while preventing the associated dangers. [143] If an AGI's primary objective is to prevent existential disasters such as human termination (which might be challenging if the Vulnerable World Hypothesis turns out to be real), [144] it could take procedures to drastically lower the dangers [143] while decreasing the effect of these steps on our quality of life.


Risks


Existential threats


AGI might represent several kinds of existential danger, which are threats that threaten "the early termination of Earth-originating smart life or the permanent and drastic damage of its capacity for preferable future development". [145] The risk of human termination from AGI has been the subject of lots of disputes, but there is also the possibility that the advancement of AGI would cause a completely flawed future. Notably, it could be used to spread and protect the set of worths of whoever develops it. If humankind still has moral blind areas similar to slavery in the past, AGI may irreversibly entrench it, avoiding ethical development. [146] Furthermore, AGI might assist in mass surveillance and indoctrination, which could be used to produce a stable repressive worldwide totalitarian program. [147] [148] There is likewise a threat for the machines themselves. If devices that are sentient or otherwise worthwhile of ethical factor to consider are mass produced in the future, participating in a civilizational path that forever neglects their welfare and interests might be an existential catastrophe. [149] [150] Considering just how much AGI might improve mankind's future and aid lower other existential risks, Toby Ord calls these existential risks "an argument for proceeding with due care", not for "abandoning AI". [147]

Risk of loss of control and human extinction


The thesis that AI postures an existential danger for humans, and that this danger requires more attention, is controversial but has actually been backed in 2023 by lots of public figures, AI scientists and CEOs of AI companies such as Elon Musk, Bill Gates, Geoffrey Hinton, Yoshua Bengio, Demis Hassabis and Sam Altman. [151] [152]

In 2014, Stephen Hawking slammed prevalent indifference:


So, facing possible futures of incalculable advantages and risks, the specialists are definitely doing whatever possible to ensure the very best outcome, right? Wrong. If an exceptional alien civilisation sent us a message saying, 'We'll show up in a few decades,' would we simply reply, 'OK, call us when you get here-we'll leave the lights on?' Probably not-but this is more or less what is occurring with AI. [153]

The prospective fate of mankind has sometimes been compared to the fate of gorillas threatened by human activities. The comparison specifies that higher intelligence permitted mankind to control gorillas, which are now vulnerable in methods that they might not have actually expected. As a result, the gorilla has ended up being an endangered species, not out of malice, but merely as a civilian casualties from human activities. [154]

The skeptic Yann LeCun thinks about that AGIs will have no desire to control humankind which we must take care not to anthropomorphize them and analyze their intents as we would for humans. He stated that people will not be "wise sufficient to create super-intelligent makers, yet ridiculously stupid to the point of offering it moronic objectives with no safeguards". [155] On the other side, the idea of instrumental convergence suggests that practically whatever their objectives, smart representatives will have factors to try to survive and acquire more power as intermediary steps to attaining these objectives. And that this does not require having feelings. [156]

Many scholars who are concerned about existential threat supporter for more research study into fixing the "control problem" to answer the concern: what types of safeguards, algorithms, or architectures can developers carry out to maximise the likelihood that their recursively-improving AI would continue to act in a friendly, rather than destructive, manner after it reaches superintelligence? [157] [158] Solving the control issue is made complex by the AI arms race (which might result in a race to the bottom of safety precautions in order to launch products before rivals), [159] and making use of AI in weapon systems. [160]

The thesis that AI can present existential threat likewise has detractors. Skeptics generally state that AGI is unlikely in the short-term, or that concerns about AGI sidetrack from other concerns connected to current AI. [161] Former Google scams czar Shuman Ghosemajumder thinks about that for lots of people outside of the technology market, existing chatbots and LLMs are already viewed as though they were AGI, causing further misunderstanding and fear. [162]

Skeptics often charge that the thesis is crypto-religious, with an irrational belief in the possibility of superintelligence replacing an irrational belief in a supreme God. [163] Some scientists think that the communication projects on AI existential danger by certain AI groups (such as OpenAI, Anthropic, DeepMind, and Conjecture) might be an at effort at regulative capture and to inflate interest in their items. [164] [165]

In 2023, the CEOs of Google DeepMind, OpenAI and Anthropic, in addition to other industry leaders and researchers, issued a joint declaration asserting that "Mitigating the danger of extinction from AI should be an international concern alongside other societal-scale risks such as pandemics and nuclear war." [152]

Mass unemployment


Researchers from OpenAI approximated that "80% of the U.S. workforce might have at least 10% of their work jobs affected by the intro of LLMs, while around 19% of employees might see at least 50% of their tasks impacted". [166] [167] They consider workplace workers to be the most exposed, for instance mathematicians, accounting professionals or web designers. [167] AGI could have a much better autonomy, capability to make choices, to user interface with other computer system tools, but likewise to manage robotized bodies.


According to Stephen Hawking, the outcome of automation on the lifestyle will depend upon how the wealth will be redistributed: [142]

Everyone can enjoy a life of elegant leisure if the machine-produced wealth is shared, or the majority of people can end up badly poor if the machine-owners successfully lobby versus wealth redistribution. Up until now, the trend seems to be towards the 2nd option, with innovation driving ever-increasing inequality


Elon Musk thinks about that the automation of society will need federal governments to adopt a universal fundamental earnings. [168]

See likewise


Artificial brain - Software and hardware with cognitive capabilities comparable to those of the animal or human brain
AI impact
AI security - Research area on making AI safe and helpful
AI positioning - AI conformance to the desired goal
A.I. Rising - 2018 film directed by Lazar Bodroža
Expert system
Automated artificial intelligence - Process of automating the application of maker learning
BRAIN Initiative - Collaborative public-private research initiative announced by the Obama administration
China Brain Project
Future of Humanity Institute - Defunct Oxford interdisciplinary research study centre
General video game playing - Ability of expert system to play different games
Generative artificial intelligence - AI system efficient in creating material in response to triggers
Human Brain Project - Scientific research task
Intelligence amplification - Use of infotech to enhance human intelligence (IA).
Machine principles - Moral behaviours of man-made devices.
Moravec's paradox.
Multi-task learning - Solving numerous device discovering jobs at the very same time.
Neural scaling law - Statistical law in artificial intelligence.
Outline of expert system - Overview of and topical guide to synthetic intelligence.
Transhumanism - Philosophical movement.
Synthetic intelligence - Alternate term for or form of expert system.
Transfer knowing - Machine learning technique.
Loebner Prize - Annual AI competitors.
Hardware for expert system - Hardware specially developed and enhanced for artificial intelligence.
Weak artificial intelligence - Form of artificial intelligence.


Notes


^ a b See listed below for the origin of the term "strong AI", and see the academic meaning of "strong AI" and weak AI in the post Chinese space.
^ AI creator John McCarthy writes: "we can not yet identify in basic what sort of computational procedures we desire to call intelligent. " [26] (For a conversation of some definitions of intelligence utilized by artificial intelligence researchers, see approach of expert system.).
^ The Lighthill report specifically slammed AI's "grandiose goals" and led the dismantling of AI research in England. [55] In the U.S., DARPA became determined to money only "mission-oriented direct research, instead of fundamental undirected research study". [56] [57] ^ As AI creator John McCarthy writes "it would be a terrific relief to the rest of the employees in AI if the creators of brand-new general formalisms would reveal their hopes in a more guarded kind than has actually in some cases been the case." [61] ^ In "Mind Children" [122] 1015 cps is used. More recently, in 1997, [123] Moravec argued for 108 MIPS which would roughly represent 1014 cps. Moravec talks in regards to MIPS, not "cps", which is a non-standard term Kurzweil presented.
^ As specified in a basic AI book: "The assertion that makers might potentially act smartly (or, perhaps much better, act as if they were intelligent) is called the 'weak AI' hypothesis by thinkers, and the assertion that devices that do so are in fact believing (as opposed to replicating thinking) is called the 'strong AI' hypothesis." [121] ^ Alan Turing made this point in 1950. [36] References


^ Krishna, Sri (9 February 2023). "What is synthetic narrow intelligence (ANI)?". VentureBeat. Retrieved 1 March 2024. ANI is created to carry out a single task.
^ "OpenAI Charter". OpenAI. Retrieved 6 April 2023. Our objective is to ensure that artificial basic intelligence advantages all of humankind.
^ Heath, Alex (18 January 2024). "Mark Zuckerberg's new goal is creating synthetic basic intelligence". The Verge. Retrieved 13 June 2024. Our vision is to construct AI that is better than human-level at all of the human senses.
^ Baum, Seth D. (2020 ). A Study of Artificial General Intelligence Projects for Ethics, Risk, and Policy (PDF) (Report). Global Catastrophic Risk Institute. Retrieved 28 November 2024. 72 AGI R&D projects were determined as being active in 2020.
^ a b c "AI timelines: What do specialists in synthetic intelligence anticipate for the future?". Our World in Data. Retrieved 6 April 2023.
^ Metz, Cade (15 May 2023). "Some Researchers Say A.I. Is Already Here, Stirring Debate in Tech Circles". The New York City Times. Retrieved 18 May 2023.
^ "AI leader Geoffrey Hinton quits Google and warns of threat ahead". The New York Times. 1 May 2023. Retrieved 2 May 2023. It is tough to see how you can prevent the bad actors from using it for bad things.
^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric (2023 ). "Sparks of Artificial General Intelligence: Early try outs GPT-4". arXiv preprint. arXiv:2303.12712. GPT-4 shows triggers of AGI.
^ Butler, Octavia E. (1993 ). Parable of the Sower. Grand Central Publishing. ISBN 978-0-4466-7550-5. All that you touch you alter. All that you alter changes you.
^ Vinge, Vernor (1992 ). A Fire Upon the Deep. Tor Books. ISBN 978-0-8125-1528-2. The Singularity is coming.
^ Morozov, Evgeny (30 June 2023). "The True Threat of Artificial Intelligence". The New York Times. The genuine threat is not AI itself but the way we deploy it.
^ "Impressed by expert system? Experts state AGI is following, and it has 'existential' dangers". ABC News. 23 March 2023. Retrieved 6 April 2023. AGI might present existential risks to humankind.
^ Bostrom, Nick (2014 ). Superintelligence: Paths, Dangers, Strategies. Oxford University Press. ISBN 978-0-1996-7811-2. The first superintelligence will be the last creation that humankind requires to make.
^ Roose, Kevin (30 May 2023). "A.I. Poses 'Risk of Extinction,' Industry Leaders Warn". The New York City Times. Mitigating the danger of extinction from AI must be a global priority.
^ "Statement on AI Risk". Center for AI Safety. Retrieved 1 March 2024. AI experts alert of threat of extinction from AI.
^ Mitchell, Melanie (30 May 2023). "Are AI's Doomsday Scenarios Worth Taking Seriously?". The New York City Times. We are far from developing makers that can outthink us in basic methods.
^ LeCun, Yann (June 2023). "AGI does not present an existential risk". Medium. There is no reason to fear AI as an existential threat.
^ Kurzweil 2005, p. 260.
^ a b Kurzweil, Ray (5 August 2005), "Long Live AI", Forbes, archived from the initial on 14 August 2005: Kurzweil describes strong AI as "machine intelligence with the full series of human intelligence.".
^ "The Age of Artificial Intelligence: George John at TEDxLondonBusinessSchool 2013". Archived from the initial on 26 February 2014. Retrieved 22 February 2014.
^ Newell & Simon 1976, This is the term they utilize for "human-level" intelligence in the physical sign system hypothesis.
^ "The Open University on Strong and Weak AI". Archived from the initial on 25 September 2009. Retrieved 8 October 2007.
^ "What is artificial superintelligence (ASI)?|Definition from TechTarget". Enterprise AI. Retrieved 8 October 2023.
^ "Expert system is changing our world - it is on all of us to make certain that it goes well". Our World in Data. Retrieved 8 October 2023.
^ Dickson, Ben (16 November 2023). "Here is how far we are to accomplishing AGI, according to DeepMind". VentureBeat.
^ McCarthy, John (2007a). "Basic Questions". Stanford University. Archived from the original on 26 October 2007. Retrieved 6 December 2007.
^ This list of intelligent qualities is based upon the subjects covered by major AI textbooks, consisting of: Russell & Norvig 2003, Luger & Stubblefield 2004, Poole, Mackworth & Goebel 1998 and Nilsson 1998.
^ Johnson 1987.
^ de Charms, R. (1968 ). Personal causation. New York: Academic Press.
^ a b Pfeifer, R. and Bongard J. C., How the body forms the way we believe: a new view of intelligence (The MIT Press, 2007). ISBN 0-2621-6239-3.
^ White, R. W. (1959 ). "Motivation reconsidered: The principle of proficiency". Psychological Review. 66 (5 ): 297-333. doi:10.1037/ h0040934. PMID 13844397. S2CID 37385966.
^ White, R. W. (1959 ). "Motivation reevaluated: The principle of competence". Psychological Review. 66 (5 ): 297-333. doi:10.1037/ h0040934. PMID 13844397. S2CID 37385966.
^ Muehlhauser, Luke (11 August 2013). "What is AGI?". Machine Intelligence Research Institute. Archived from the original on 25 April 2014. Retrieved 1 May 2014.
^ "What is Artificial General Intelligence (AGI)?|4 Tests For Ensuring Artificial General Intelligence". Talky Blog. 13 July 2019. Archived from the original on 17 July 2019. Retrieved 17 July 2019.
^ Kirk-Giannini, Cameron Domenico; Goldstein, Simon (16 October 2023). "AI is closer than ever to passing the Turing test for 'intelligence'. What takes place when it does?". The Conversation. Retrieved 22 September 2024.
^ a b Turing 1950.
^ Turing, Alan (1952 ). B. Jack Copeland (ed.). Can Automatic Calculating Machines Be Said To Think?. Oxford: Oxford University Press. pp. 487-506. ISBN 978-0-1982-5079-1.
^ "Eugene Goostman is a real kid - the Turing Test says so". The Guardian. 9 June 2014. ISSN 0261-3077. Retrieved 3 March 2024.
^ "Scientists dispute whether computer system 'Eugene Goostman' passed Turing test". BBC News. 9 June 2014. Retrieved 3 March 2024.
^ Jones, Cameron R.; Bergen, Benjamin K. (9 May 2024). "People can not differentiate GPT-4 from a human in a Turing test". arXiv:2405.08007 [cs.HC]
^ Varanasi, Lakshmi (21 March 2023). "AI designs like ChatGPT and GPT-4 are acing whatever from the bar test to AP Biology. Here's a list of tough exams both AI variations have passed". Business Insider. Retrieved 30 May 2023.
^ Naysmith, Caleb (7 February 2023). "6 Jobs Artificial Intelligence Is Already Replacing and How Investors Can Take Advantage Of It". Retrieved 30 May 2023.
^ Turk, Victoria (28 January 2015). "The Plan to Replace the Turing Test with a 'Turing Olympics'". Vice. Retrieved 3 March 2024.
^ Gopani, Avi (25 May 2022). "Turing Test is unreliable. The Winograd Schema is obsolete. Coffee is the answer". Analytics India Magazine. Retrieved 3 March 2024.
^ Bhaimiya, Sawdah (20 June 2023). "DeepMind's co-founder recommended evaluating an AI chatbot's capability to turn $100,000 into $1 million to determine human-like intelligence". Business Insider. Retrieved 3 March 2024.
^ Suleyman, Mustafa (14 July 2023). "Mustafa Suleyman: My brand-new Turing test would see if AI can make $1 million". MIT Technology Review. Retrieved 3 March 2024.
^ Shapiro, Stuart C. (1992 ). "Expert System" (PDF). In Stuart C. Shapiro (ed.). Encyclopedia of Expert System (Second ed.). New York: John Wiley. pp. 54-57. Archived (PDF) from the original on 1 February 2016. (Section 4 is on "AI-Complete Tasks".).
^ Yampolskiy, Roman V. (2012 ). Xin-She Yang (ed.). "Turing Test as a Specifying Feature of AI-Completeness" (PDF). Artificial Intelligence, Evolutionary Computation and Metaheuristics (AIECM): 3-17. Archived (PDF) from the original on 22 May 2013.
^ "AI Index: State of AI in 13 Charts". Stanford University Human-Centered Expert System. 15 April 2024. Retrieved 27 May 2024.
^ Crevier 1993, pp. 48-50.
^ Kaplan, Andreas (2022 ). "Artificial Intelligence, Business and Civilization - Our Fate Made in Machines". Archived from the original on 6 May 2022. Retrieved 12 March 2022.
^ Simon 1965, p. 96 priced estimate in Crevier 1993, p. 109.
^ "Scientist on the Set: An Interview with Marvin Minsky". Archived from the initial on 16 July 2012. Retrieved 5 April 2008.
^ Marvin Minsky to Darrach (1970 ), priced estimate in Crevier (1993, p. 109).
^ Lighthill 1973; Howe 1994.
^ a b NRC 1999, "Shift to Applied Research Increases Investment".
^ Crevier 1993, pp. 115-117; Russell & Norvig 2003, pp. 21-22.
^ Crevier 1993, p. 211, Russell & Norvig 2003, p. 24 and see likewise Feigenbaum & McCorduck 1983.
^ Crevier 1993, pp. 161-162, 197-203, 240; Russell & Norvig 2003, p. 25.
^ Crevier 1993, pp. 209-212.
^ McCarthy, John (2000 ). "Respond to Lighthill". Stanford University. Archived from the original on 30 September 2008. Retrieved 29 September 2007.
^ Markoff, John (14 October 2005). "Behind Artificial Intelligence, a Squadron of Bright Real People". The New York City Times. Archived from the initial on 2 February 2023. Retrieved 18 February 2017. At its low point, some computer system scientists and software application engineers prevented the term expert system for fear of being viewed as wild-eyed dreamers.
^ Russell & Norvig 2003, pp. 25-26
^ "Trends in the Emerging Tech Hype Cycle". Gartner Reports. Archived from the original on 22 May 2019. Retrieved 7 May 2019.
^ a b Moravec 1988, p. 20
^ Harnad, S. (1990 ). "The Symbol Grounding Problem". Physica D. 42 (1-3): 335-346. arXiv: cs/9906002. Bibcode:1990 PhyD ... 42..335 H. doi:10.1016/ 0167-2789( 90 )90087-6. S2CID 3204300.
^ Gubrud 1997
^ Hutter, Marcus (2005 ). Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Texts in Theoretical Computer Science an EATCS Series. Springer. doi:10.1007/ b138233. ISBN 978-3-5402-6877-2. S2CID 33352850. Archived from the original on 19 July 2022. Retrieved 19 July 2022.
^ Legg, Shane (2008 ). Machine Super Intelligence (PDF) (Thesis). University of Lugano. Archived (PDF) from the original on 15 June 2022. Retrieved 19 July 2022.
^ Goertzel, Ben (2014 ). Artificial General Intelligence. Lecture Notes in Computer Technology. Vol. 8598. Journal of Artificial General Intelligence. doi:10.1007/ 978-3-319-09274-4. ISBN 978-3-3190-9273-7. S2CID 8387410.
^ "Who created the term "AGI"?". goertzel.org. Archived from the original on 28 December 2018. Retrieved 28 December 2018., through Life 3.0: 'The term "AGI" was popularized by ... Shane Legg, Mark Gubrud and Ben Goertzel'
^ Wang & Goertzel 2007
^ "First International Summer School in Artificial General Intelligence, Main summer season school: June 22 - July 3, 2009, OpenCog Lab: July 6-9, 2009". Archived from the original on 28 September 2020. Retrieved 11 May 2020.
^ "Избираеми дисциплини 2009/2010 - пролетен триместър" [Elective courses 2009/2010 - spring trimester] Факултет по математика и информатика [Faculty of Mathematics and Informatics] (in Bulgarian). Archived from the initial on 26 July 2020. Retrieved 11 May 2020.
^ "Избираеми дисциплини 2010/2011 - зимен триместър" [Elective courses 2010/2011 - winter season trimester] Факултет по математика и информатика [Faculty of Mathematics and Informatics] (in Bulgarian). Archived from the initial on 26 July 2020. Retrieved 11 May 2020.
^ Shevlin, Henry; Vold, Karina; Crosby, Matthew; Halina, Marta (4 October 2019). "The limitations of machine intelligence: Despite development in device intelligence, synthetic basic intelligence is still a significant challenge". EMBO Reports. 20 (10 ): e49177. doi:10.15252/ embr.201949177. ISSN 1469-221X. PMC 6776890. PMID 31531926.
^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric; Kamar, Ece; Lee, Peter; Lee, Yin Tat; Li, Yuanzhi; Lundberg, Scott; Nori, Harsha; Palangi, Hamid; Ribeiro, Marco Tulio; Zhang, Yi (27 March 2023). "Sparks of Artificial General Intelligence: Early explores GPT-4". arXiv:2303.12712 [cs.CL]
^ "Microsoft Researchers Claim GPT-4 Is Showing "Sparks" of AGI". Futurism. 23 March 2023. Retrieved 13 December 2023.
^ Allen, Paul; Greaves, Mark (12 October 2011). "The Singularity Isn't Near". MIT Technology Review. Retrieved 17 September 2014.
^ Winfield, Alan. "Expert system will not become a Frankenstein's beast". The Guardian. Archived from the initial on 17 September 2014. Retrieved 17 September 2014.
^ Deane, George (2022 ). "Machines That Feel and Think: The Role of Affective Feelings and Mental Action in (Artificial) General Intelligence". Artificial Life. 28 (3 ): 289-309. doi:10.1162/ artl_a_00368. ISSN 1064-5462. PMID 35881678. S2CID 251069071.
^ a b c Clocksin 2003.
^ Fjelland, Ragnar (17 June 2020). "Why general synthetic intelligence will not be recognized". Humanities and Social Sciences Communications. 7 (1 ): 1-9. doi:10.1057/ s41599-020-0494-4. hdl:11250/ 2726984. ISSN 2662-9992. S2CID 219710554.
^ McCarthy 2007b.
^ Khatchadourian, Raffi (23 November 2015). "The Doomsday Invention: Will expert system bring us paradise or destruction?". The New Yorker. Archived from the original on 28 January 2016. Retrieved 7 February 2016.
^ Müller, V. C., & Bostrom, N. (2016 ). Future development in expert system: A survey of skilled viewpoint. In Fundamental concerns of synthetic intelligence (pp. 555-572). Springer, Cham.
^ Armstrong, Stuart, and Kaj Sotala. 2012. "How We're Predicting AI-or Failing To." In Beyond AI: Artificial Dreams, modified by Jan Romportl, Pavel Ircing, Eva Žáčková, Michal Polák and Radek Schuster, 52-75. Plzeň: University of West Bohemia
^ "Microsoft Now Claims GPT-4 Shows 'Sparks' of General Intelligence". 24 March 2023.
^ Shimek, Cary (6 July 2023). "AI Outperforms Humans in Creativity Test". Neuroscience News. Retrieved 20 October 2023.
^ Guzik, Erik E.; Byrge, Christian; Gilde, Christian (1 December 2023). "The creativity of devices: AI takes the Torrance Test". Journal of Creativity. 33 (3 ): 100065. doi:10.1016/ j.yjoc.2023.100065. ISSN 2713-3745. S2CID 261087185.
^ Arcas, Blaise Agüera y (10 October 2023). "Artificial General Intelligence Is Already Here". Noema.
^ Zia, Tehseen (8 January 2024). "Unveiling of Large Multimodal Models: Shaping the Landscape of Language Models in 2024". Unite.ai. Retrieved 26 May 2024.
^ "Introducing OpenAI o1-preview". OpenAI. 12 September 2024.
^ Knight, Will. "OpenAI Announces a New AI Model, Code-Named Strawberry, That Solves Difficult Problems Step by Step". Wired. ISSN 1059-1028. Retrieved 17 September 2024.
^ "OpenAI Employee Claims AGI Has Been Achieved". Orbital Today. 13 December 2024. Retrieved 27 December 2024.
^ "AI Index: State of AI in 13 Charts". hai.stanford.edu. 15 April 2024. Retrieved 7 June 2024.
^ "Next-Gen AI: OpenAI and Meta's Leap Towards Reasoning Machines". Unite.ai. 19 April 2024. Retrieved 7 June 2024.
^ James, Alex P. (2022 ). "The Why, What, and How of Artificial General Intelligence Chip Development". IEEE Transactions on Cognitive and Developmental Systems. 14 (2 ): 333-347. arXiv:2012.06338. doi:10.1109/ TCDS.2021.3069871. ISSN 2379-8920. S2CID 228376556. Archived from t

Kommentarer