{"id":13256,"date":"2023-05-25T12:35:10","date_gmt":"2023-05-25T02:35:10","guid":{"rendered":"https:\/\/rationalemagazine.com\/?p=13256"},"modified":"2023-05-25T12:35:10","modified_gmt":"2023-05-25T02:35:10","slug":"the-impact-and-implications-of-large-language-models","status":"publish","type":"post","link":"https:\/\/rationalemagazine.com\/index.php\/2023\/05\/25\/the-impact-and-implications-of-large-language-models\/","title":{"rendered":"The impact and implications of large language models"},"content":{"rendered":"

Large language models (LLMs) such as ChatGPT and Google Bard have attracted a great deal of publicity in recent months due to their impressive abilities to engage in conversation, answer open-ended questions, find information, write essays, generate computer code, produce poetry, and solve certain problems.<\/span><\/p>\n

The improvement of these systems has generated <\/span>significant concern<\/span><\/a> regarding the potential harms of such technology. Various potential risks have been raised, including use of LLMs to rapidly <\/span>spread misinformation<\/span><\/a>, rapid <\/span>automation<\/span><\/a> of large numbers of jobs, and artificial agents behaving in unexpected ways or even <\/span>escaping human control<\/span><\/a> entirely.<\/span><\/p>\n

A major driving force underpinning these concerns is the rapid rate of improvement of LLMs, and the belief that this rate of progress will soon lead to systems that exceed human capabilities in many tasks, leading to drastically disruptive effects on the economy and society.<\/span><\/p>\n

In this essay, I will argue that such fears are exaggerated. While there are legitimate concerns about the safety and reliability of LLMs, I do not think it is likely that such systems will soon reach human levels of intelligence or capability in a broad range of tasks.<\/span><\/p>\n

Instead, I argue that such systems have intrinsic limitations which cannot be overcome within the existing development paradigm. Also, continual growth in capabilities based on increasing the number of parameters and size of the training data will only continue for a few more years before running its course.<\/span><\/p>\n

I also argue that the adoption of such systems will be slow, occurring over years to decades rather than months to years (as <\/span>some have argued<\/span><\/a>). Therefore, their impacts on society and the economy will be more gradual and evolutionary rather than sudden and revolutionary.\u00a0<\/span><\/p>\n

Current <\/span>LLMs<\/span><\/a> are based on the <\/span>transformer architecture<\/span><\/a>. These are very large neural networks which are trained on huge corpuses of data, most of which is from the internet. The models are usually trained to predict the next word in a sentence, and during training they learn complex statistical associations between words in natural language.<\/span><\/p>\n

Recently, OpenAI has extended this framework by adding a technique called <\/span>Reinforcement Learning from Human Feedback<\/span><\/a> (RLHF). This involves presenting queries and their corresponding LLM outputs to humans, who then provide ratings as to the quality of the responses.\u00a0<\/span><\/p>\n

These ratings are then used to fine-tune the language model, altering its output to improve its ratings from human feedback. This technique has enabled language models to produce output that is more useful to humans, and has improved the performance of language models as chatbots.\u00a0<\/span><\/p>\n

The OpenAI team has also made other additions and modifications to its newest model (GPT-4) to improve its capabilities as a chatbot, though very few public details are available about this.<\/span><\/p>\n

Judging by the number of contributors to the <\/span>GPT-4 paper<\/span><\/a> (which lists 93 \u2018core contributors\u2019 and hundreds of other contributors) relative to previous <\/span>GPT-3 paper<\/span><\/a> (which lists only 31 authors), it appears that OpenAI has devoted a lot of time to adjusting, augmenting, and modifying the model in various ways.<\/span><\/p>\n

We know that systems have been put in place to filter out queries likely to lead to harmful or offensive results. There is also <\/span>evidence<\/span><\/a> that GPT-4 has a limited ability to check for faulty assumptions in the queries or instructions it is given, though it is unclear how this has been done. Nonetheless, it appears that extensive development work has been done beyond the initial stage of training the transformer on a large text corpus.<\/span><\/p>\n

In my view, the fact that such extensive augmentations and modifications are necessary is an indication of the underlying weaknesses and limitations of the transformer architecture. These models learn complex associations between words, but do not form the same structured, flexible, multimodal representations of word meaning as humans do.<\/span><\/p>\n

As such, they <\/span>do not truly \u2018understand\u2019 language<\/span><\/a> in the same sense as humans can. For many applications, this does not matter. But in other cases it can manifest in extremely bizarre behaviour, including models accepting absurd premises, making <\/span>faulty inferences<\/span><\/a>, making <\/span>contradictory statements<\/span><\/a>, and failing to incorporate information that is provided.<\/span><\/p>\n

A related issue is the known tendency of LLMs to \u2018<\/span>hallucinate<\/span><\/a>\u2019, making up facts, information, or non-existent libraries of computer code when giving responses. I dislike the term \u2018hallucination\u2019 because it implies there is some fundamental distinction between veridical knowledge that the LLM has correctly learned and hallucinations, which it simply makes up.<\/span><\/p>\n

In my view, the fact that such extensive augmentations and modifications are necessary is an indication of the underlying weaknesses and limitations of the transformer architecture.<\/strong><\/p><\/blockquote>\n

In fact, there is no such distinction, because <\/span>LLMs do not form memories<\/span><\/a> of events or facts in the way humans do. All they are capable of is storing complex statistical associations in their billions of learned parameters.<\/span><\/p>\n

When the model produces some string of words as an output, this is equally the product of its internal learned parameters regardless of whether humans would evaluate the string as true or false.<\/span><\/p>\n

Furthermore, an LLM has no notion of truth or falsity; it simply learns word associations. (Here I am ignoring the possibility that GPT-4 may be augmented with capabilities beyond its basic transformer architecture, since there is no public information about this. And, at any rate, the underlying architecture is still a transformer model). As such, the problem of \u2018hallucinations\u2019 is not some teething issue or minor annoyance, but is intrinsic to the architecture and method of training of LLMs.<\/span><\/p>\n

Of course, various proposals exist for how to mitigate this limitation, such as augmenting LLMs with curated datasets of encyclopaedic facts or <\/span>common-sense knowledge<\/span><\/a>. While promising, such proposals are not new and <\/span>face many problems<\/span><\/a> of their own right. While they may be successful in the long run, I do not believe there is any simple or easily implemented solution to the problem of \u2018hallucinations\u2019 in LLMs.<\/span><\/p>\n

Another core limitation of LLMs which has been the focus of extensive research is their difficulty in exhibiting compositionality. This refers to the ability to combine known elements in novel ways by following certain abstract rules.\u00a0<\/span><\/p>\n

Many cognitive scientists have argued that compositionality is a <\/span>critical component<\/span><\/a> of the human ability to understand novel sentences with combinations of words and ideas never previously encountered.<\/span><\/p>\n

Prior to the release of GPT-4, the best transformer models still <\/span>struggled to perform many compositional<\/span><\/a> tasks, often only succeeding when augmented with symbolic components (which is difficult to scale to real-world tasks), or when given special task-specific training.\u00a0<\/span><\/p>\n

At the time of writing, I am not aware of GPT-4 having been subjected to these types of tests. Although I anticipate it would outperform most existing models \u2013 given that it shares the same transformer architecture \u2013 I doubt it will be able to completely solve the problem of compositionality.<\/span><\/p>\n

The underlying limitation is that transformer-based language models do not learn explicit symbolic representations, and hence struggle to generalise appropriately in accordance with systematic rules.\u00a0<\/span><\/p>\n

There have also been efforts to circumvent some of these limitations and use LLMs for a wider range of tasks by developing them into a partially autonomous agent. The approach is to chain together a series of instructions, allowing the model to step through subcomponents of a task and reason its way to the desired conclusion.<\/span><\/p>\n

One such project called <\/span>Auto-GPT<\/span><\/a> involves augmenting GPT with the ability to read and write from external memory, and allowing it access to various external software packages through their application programming interfaces (APIs).<\/span><\/p>\n

It is too early to say what will become of such projects, though early investigations indicate some promising results but also plenty of <\/span>difficulties<\/span><\/a>. In particular, the model often gets stuck in loops, fails to correctly incorporate contextual knowledge to constrain solutions to the problem, and has no ability to generalise results to similar future problems.<\/span><\/p>\n

Such difficulties illustrate that LLMs are not designed to be general purpose agents, and hence lack many cognitive faculties such as planning, learning, decision making, or symbolic reasoning.<\/span><\/p>\n

Furthermore, it is exceedingly unlikely that simply \u2018plugging in\u2019 various components to an LLM in an ad hoc manner will result in an agent capable of performing competently in a diverse range of environments.<\/span><\/p>\n

The way the components are connected and interact is absolutely crucial to the overall capabilities of the system. The structure of the different cognitive components of an agent is called a <\/span>cognitive architecture<\/span><\/a>, and there has been decades of research into this topic in both cognitive psychology and computer science.<\/span><\/p>\n

As such, I think it is na\u00efve to believe that such research will be rendered irrelevant or obsolete by the simple expedient of augmenting LLMs with a few additional components. Instead, I expect that LLMs will form one component of many that will need to be incorporated into a truly general-purpose intelligent system, one which will likely take decades of further research to develop.\u00a0<\/span><\/p>\n

Recent improvements in LLMs have primarily occurred as a result of dramatic increases in both the number of model parameters and the size of the training datasets. This has led to a rapid increase in training costs, largely due to the electricity usage and rental or opportunity cost of the required hardware. For example, the cost of training GPT-3 was probably <\/span>several million dollars<\/span><\/a>, compared to over <\/span>one hundred million<\/span><\/a> for GPT-4.<\/span><\/p>\n

Assuming current growth rates continue, within about five years further increasing model size will become infeasible even for the biggest governments and tech firms, as training costs will reach tens of billions of dollars.\u00a0<\/span><\/p>\n

The next few years will be a critical period for LLMs, in which there will be much experimentation and failed attempts as companies compete to find the best way to deploy<\/a> the technology.<\/strong><\/p><\/blockquote>\n

Separately from the issue of training cost, there is also the question of the availability of training data. Existing models require enormous training datasets, with the size increasing exponentially from one iteration to the next. For example, GPT-3 was trained on a primary corpus of <\/span>300 billion words<\/span><\/a> derived from the internet. Based on historical trends, <\/span>Epoch estimates<\/span><\/a> that high quality language data will be exhausted by 2024 or 2025, and low quality data by 2032.<\/span><\/p>\n

I am not arguing here that the development of LLMs will cease within five years or that further improvements are impossible. Rather my point is that the primary method by which improvements have been achieved over the past five years will cease to be feasible. As such, we cannot expect current rates of progress to continue indefinitely. Similar views have been expressed by other researchers, including <\/span>Ben Goertzel<\/span><\/a>, <\/span>Gary Marcus<\/span><\/a>, and <\/span>Sam Altman<\/span><\/a>.<\/span><\/p>\n

In light of these considerations, along with the intrinsic limitations discussed above, I do not think it is plausible that LLMs will reach or exceed human performance in a wide range of tasks in the near future.<\/span><\/p>\n

The next few years will be a critical period for LLMs, in which there will be much experimentation and failed attempts as companies compete to find the <\/span>best way to deploy<\/span><\/a> the technology. It will take considerable time and effort to turn LLMs into a viable product, and even longer to adapt its use to various speciality applications and for the technology to become <\/span>widely adopted<\/span><\/a>.<\/span><\/p>\n

Many companies and organisations will seek ways to use LLMs to augment their existing internal processes and procedures, which also will take a great deal of time and trial and error.<\/span><\/p>\n

Contrary to what some have implied, no new technology can ever simply be \u2018plugged in\u2019 to existing processes without substantial change or adaptation. Just as automobiles, computers, and the internet took decades to have major economic and social impacts, so too I expect LLMs will take decades to have such impacts. Yet other technologies, such as <\/span>nuclear fusion<\/span><\/a>, <\/span>reusable launch vehicles<\/span><\/a>, commercial <\/span>supersonic flight<\/span><\/a>, are still yet to achieve their promised substantial impact.\u00a0<\/span><\/p>\n

One of the major limitations of using existing LLMs is their unreliability. No important processes can currently be trusted to LLMs, because we have very little understanding of how they work, limited knowledge of the limits of their capabilities, and a poor understanding of how and when they fail. They are able to perform impressive feats, but then fail in particularly unexpected and surprising ways.\u00a0<\/span><\/p>\n

Unpredictability and unreliability both make it very difficult to use LLMs for many business or government tasks. Of course, humans regularly make mistakes, but human capabilities and fallibilities are better understood than those of LLMs, and existing political, economic, and governance systems have been developed over many decades to manage human mistakes and imperfections.<\/span><\/p>\n

I expect it will similarly take many years to build systems to effectively work around the limitations of LLMs and achieve sufficient reliability for widespread deployment.<\/span><\/p>\n

It is also valuable to take a historical perspective, as the field of artificial intelligence has seen numerous examples of excessive hype and inflated expectations. In the late 1950s and early 1960s, there was a wave of enthusiasm about the promise of logic-based systems and automated reasoning, which were thought to be capable of overtaking humans in many tasks within a matter of years.<\/span><\/p>\n

The failure of many of these predictions led to the <\/span>first \u2018AI winter<\/span><\/a>\u2019 of the 1970s. The 1980s saw a resurgence of interest in AI, this time based on new approaches such as expert systems, the backpropagation algorithm, and initiatives such as Japan’s Fifth Generation computer initiative. Underperformance of these systems and techniques led to <\/span>another \u2018AI winter<\/span><\/a>\u2019 in the 1990s and early 2000s.<\/span><\/p>\n

The most recent resurgence of interest in AI has largely been driven by breakthroughs in machine learning and the availability of much larger sources of data for training. Progress in the past 15 years has been rapid and impressive, but even so there have been numerous instances of inflated expectations and failed promises.\u00a0<\/span><\/p>\n

IBM\u2019s Watson system which won <\/span>Jeopardy!<\/span><\/i> in 2011 was heralded by IBM as a critical breakthrough in AI research, but subsequently they spent years attempting to adapt the system for use in medical diagnosis with <\/span>little success<\/span><\/a>.<\/span><\/p>\n

Self-driving cars developed by Google attracted <\/span>substantial publicity in 2012<\/span><\/a> with their ability to drive autonomously on public roads with minimal human intervention. But, a decade later, there remain <\/span>considerable challenges<\/span><\/a> in closing the last few small portions of the journey where humans still need to take over.\u00a0<\/span><\/p>\n

\"\"<\/a><\/p>\n

While such comparisons can never be definitive, I believe these historical precedents should temper our expectations about the rate of progress of the latest set of techniques in artificial intelligence research.<\/span><\/p>\n

In conclusion, LLMs have intrinsic limitations which are unlikely to be resolved without fundamental new paradigms. The increasing costs of training and limited stock of quality training data will mean that growth of LLMs at present rates will not be able to continue for more than a few years. Furthermore, historical parallels indicate that it will take years for LLMs to become widely adopted and integrated into existing economic and social processes.<\/span><\/p>\n

Overall, there is little reason to believe that LLMs are likely to exceed human capabilities in a wide range of tasks within a few years, or displace large fractions of the workforce. These outcomes may occur in 30 or 50 years time, but almost certainly not within the next five or 10 years \u2013 and not solely due to the continued development of LLMs.<\/span><\/p>\n

While there are legitimate concerns and problems associated with the rapid improvement of LLMs, we should not be distracted by inflated concerns about catastrophic impacts in the near term.<\/span><\/p>\n

If you wish to republish this original article, please attribute to\u00a0<\/i><\/b>Rationale<\/i><\/b><\/a>.\u00a0<\/i><\/b>Click here<\/i><\/b><\/a>\u00a0to find out more about republishing under Creative Commons.<\/i><\/b><\/p>\n

Photo by Mojahid Mottakin<\/a><\/i><\/b>\u00a0on Unsplash.<\/i><\/b><\/p>\n","protected":false},"excerpt":{"rendered":"

Large language models (LLMs) such as ChatGPT and Google Bard have attracted a great deal of publicity in recent months<\/p>\n","protected":false},"author":139,"featured_media":13257,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[65],"tags":[562],"coauthors":[156],"class_list":["post-13256","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-science-health","tag-artificial-intelligence"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/posts\/13256","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/users\/139"}],"replies":[{"embeddable":true,"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/comments?post=13256"}],"version-history":[{"count":2,"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/posts\/13256\/revisions"}],"predecessor-version":[{"id":13259,"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/posts\/13256\/revisions\/13259"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/media\/13257"}],"wp:attachment":[{"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/media?parent=13256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/categories?post=13256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/tags?post=13256"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/rationalemagazine.com\/index.php\/wp-json\/wp\/v2\/coauthors?post=13256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}