Latest Blog

Living with the Enemy – Staying Human in the Era of Superintelligent Machines

By Rohit Talwar, Steve Wells and Alexandra Whittington

It has been fascinating to watch the rhetoric of technology companies evolve in the last 18 months. From heralding the potential of technologies such as Artificial Intelligence (AI) to automate processes and reduce staff costs, they have rapidly evolved to a new story line of “augmented intelligence” – arguing that the machines are here to serve not replace us. Why? The reality is that throughout history, turkeys have steadfastly refused to vote for an early Christmas. So, from business leaders to front line employees, the penny is dropping with an ever-louder and more resonant thud. The realization is growing that – unlike previous industrial revolutions and waves of technological change – this time it’s different. Hence, in the face of rising concerns about the impact on jobs and society, the technologists are in damage limitation mode pretending that their Ferraris are just pushbikes with the addition of a shopping basket and mud guards.

So, what’s really going on? Whilst it is far too early to know or predict the true potential and resulting impact of AI, it is clear that it will have widespread applications and deep ramifications across society. These include scenarios such as the end of jobs as a mainstream income-generating activity, the rise of state guaranteed incomes, the emergence of fully automated Decentralized Autonomous Organizations (DAOs) with no employees, and a massive shift in societal norms and culture if leisure becomes the prime use of our time. To help explore these next waves of change, below we address six key questions on the rise of these disruptive exponential technologies, the impact on humanity and how we can ensure a very human future in the age of superintelligent machines.

Q1: How might we prepare for changing relationship between humans and machines over the next 10-20 years?

As we look at the changes shaping our world and the pace of technological advancement, some very big questions start to arise:

  • Are humans irrelevant to the future of business?
  • What role should humans play when machines can outperform them in most tasks?
  • How should society prepare for an unknowable future?

We see five important dimensions that we must address as part of securing humanity’s future in an automated world and ensuring that the advances in technology are used to serve humanity – not replace it.

  1. Reframing Society – We are reaching a truly dramatic point in human history where a number of exponential technologies are being combined to deliver radical performance improvements. A powerful mix of unleashed imaginations applied to disruptive technologies is catalyzing a possibility revolution across every aspect of human life, society, government, and business. As a result, in the next few years, society will be challenged to confront fundamental issues that go to the core of what it means to be human. Advances in science and technology will test every assumption we have about how our world works and the purpose of humans within it. For example, AI already outperforms humans in many domains, and the possibility of “artificial superintelligence”, or constantly learning and evolving systems, could result in machines capable of overtaking human capabilities – ultimately even in so-called soft skills such as empathy, intuition, and creativity.
  2. Humanity 2.0 – Advances in cognitive enhancement drugs and “nootropic” supplements, electronic brain stimulation techniques, genetics, age extension treatments, 3D printed limbs and organs, and body worn exoskeletons, have given rise to the notion of enhancing the human brain and body well beyond the limits of natural evolutionary processes. Indeed, many leaders in the field of AI are fierce advocates of “Transhumanism” as the next stage of human evolution. They argue that if humans want to keep up with AI, we ourselves will have to become machines – embedding technology in our brains and bodies to give us similar levels of processing power. So, is there a meaningful future for version 1.0 humans in this brave new and enhanced world that the techno-progressives would have us believe is the only viable way forward for humanity? Will we have to enhance ourselves if we want to be considered for one of the potentially declining number of jobs that might be available?
  3. The Risks of Automation – The challenge here lies in our choices as decision makers and the value we place on human attributes that machines cannot as yet replicate. Clearly, automation has many benefits such as cost efficiency, consistency, speed, and accuracy. Many firms will inevitably choose to place their faith in computer systems, automating wherever possible. Such a philosophy is common in new technology ventures where the heart of the business is embedded in its code. Some are already creating DAO – entities that have no employees and exist entirely in software. The potential rewards of widespread digitization of an enterprise are well-covered in the business media, but what isn’t talked about enough is the spectrum of risks presented by automation, especially to well-established organizations. Companies run the risk of dehumanizing and becoming identical to others in their industry – losing whatever their distinctive edge might be and commoditizing themselves in the process. Furthermore, the more we choose to embed all that we do in software, the easier it becomes for competitors to replicate our offering and go a step further at a slightly lower price – locking us into deadly race to the bottom on prices, revenues, and profitability.
  4. What Differentiates Humans from Machines? – The challenge is to harness AI and other disruptive technologies such as robotics, cloud computing, the Internet of Things (IoT), blockchain, and hyperconnectivity as power tools to support and unleash human potential. At least for some time to come, what differentiates a company will be very human characteristics – the quality of its ideas, strategies and business models; its community relations; the ability to spot and exploit opportunities or address changing situations, problems and risks quickly; handling exceptional customer needs; creating new products and services; building deep connections within and outside the organization; how it navigates external developments such as regulatory requirements; and how well it manages change. These still remain very human traits which machines cannot as yet replicate. New technologies can play a powerful role in supporting the people performing these tasks and automating the more routine work to free up time for us to undertake these higher level human functions. Organizations that see AI as simply a way to cut back on staffing are missing the point and potentially short-changing their future.
  5. Unleashing Human Potential – Artificial intelligence is increasing business productivity, knowledge, and efficiency, but humans cannot be written off just yet. For example, In the insurance industry, whilst chatbots are emerging at the customer interface, there is a concern that AI is not yet at the point where machines can respond appropriately to distressed customers, an unfortunately common emotional state due to the nature of matters insurance companies deal with. Artificial intelligence offers a chance to re-humanize the workforce by providing more time to use our talents and softer skills and emotional intelligence while offloading less sensitive tasks to machines. Obviously, we will need training and support to help us step into these intellectually more demanding roles and to develop our capacity for empathy, sensitivity, compassion, creativity, and intuitive listening.
Q2: How might the new technologies impact the workplace and what new job opportunities might arise?

As individuals, managers, leaders, investors, and politicians we crave certainty and predictability. We want the future served up to us on a plate with the timelines, impacts, and solutions clearly defined. Reality is far messier and changes constantly – the only certainties are that i) ignoring the emerging future will store up problems; and ii) trying to apply yesterday’s or today’s solutions to the future’s challenges will almost certainly fail. What we do know is that the situation will evolve rapidly as the pace of technology development and adoption quickens and businesses seek to act faster to take advantage of what’s on offer and respond to potential competitive threats. A wide range of professions from sales person and school teacher to investment banker, risk assessor, claims analyst, plumber, and bus driver will see technology emerge that can enhance or even replace their roles.

Within five years, it is reasonable to foresee quite significant shifts in the types of jobs available, the skills levels required, and a shortening duration for those roles. On a ten-year timeframe, we could reasonably expect to see widespread automation, a dramatic reduction of the jobs that exist today, new roles emerging in new firms and in existing businesses as they seek to stay competitive. Educationally, a degree could become a minimum entry requirement for 80 per cent or more of all new jobs.

So, what about the messy middle between here and the end of the next decade? In the short term, the picture will be confused – certain firms and industries will accelerate rapidly towards an “employee light” model. Other sectors will see temporary skills shortages until the processes become more automated and the machines learn to code themselves. In professions ranging from machine learning specialists to quantity surveyors – we can see a near-term skills shortage with supply lagging demand. This represents a relatively short window of opportunity to retrain people for these in-demand roles. However, as the process of automation accelerates, and the way we work evolves over the next 5-10 years, we might see these skill shortages erode and the emergence of very different ways of achieving an outcome.

Within a few years, an autonomous vehicle might automatically fine its driver should they choose to take the wheel while drunk, or override the speed limit. The vehicle might also self-insure – sharing the risk across the pool of autonomous cars on the road. These smart cars might also drive themselves to the shop for repairs – carried out by a team of robots and drones. These changes wouldn’t so much re-engineer the work of solicitors, courtrooms, garages, and insurance firms – rather the activities, associated tasks, and related jobs might be eliminated completely.

New jobs will arise with the emergence of new activities, businesses, and sectors. Human augmentation will require a range of new skills, possibly combined into hybrid roles that draw on chemistry, biology, electro-mechanical engineering, psychology, and counselling. Highly trained workers will also be required in sectors such as smart materials, 3D/4D printing, autonomous car manufacture, superfast construction, environmental protection and remediation, renewable energy, and care of the elderly. In insurance, the skills of the next generation risk assessor will need to encompass a wider range of disciplines to handle the new fields of science and technology coming to market. At a more fundamental level, we could see a rise in teacher numbers if countries see education as a priority. In parallel, the opportunities in basic and applied R&D could blossom if nations and firms increase their research investments in search of future growth. We could also see a massive growth in small businesses and mentoring roles as people seek to take control of their own destiny. Finally, the stress associated with job displacement due to technology could result in a growing need both for mental health support for people whilst still in the job and for care in the community for those with mental health issues resulting from the loss of their job.

One of the most important things to keep in mind is that there could be many new definitions of the term “job” in the next 5-20 years. A job today is still a fundamental assumption and organizing principle in most Western nations – even if it is being eroded, governments still plan on that basis. A job today is a means to earn money by achieving a set of given tasks. For some it is more – a calling to fulfill one’s purpose and give meaning and structure to our lives. For others, it is a means to an end – be that paying for our next meal or providing the money to realize our materialistic, experiential, or spiritual desires.

So, as work is gradually and then more rapidly automated away, what becomes of the job? What might a job look like in 2027? Will it still be a “production” role delivering measurable daily outputs, or will a job imply a more creative human activity? Will it still be what people do all day? Conceivably, AI could remove aspects of jobs that tend to be considered “work” while emphasizing the parts of a job that make it a social and enriching activity. Will we have moved to a guaranteed or universal basic income (UBI) – with people having the choice over where they spend their time, from being a server in a restaurant to taking part in community building restoration projects? The link between how we spend our time and the income we receive might be broken in less than a decade, meaning people could have more autonomy over how they use their time and energy than ever before.

The technology we adopt today will also allow companies to increase their options in terms of achieving outcomes. While Company A might use AI to reduce the size and budget of their legal department, they might in turn boost their investment in the IT and HR departments to ensure they have the right technological capacities and that the lawyers and others they hire are absolutely the right fit. Company B might implement AI to reduce the number of customer service calls routed to human operators, but they could re-invest the salary savings in bringing in trainers and facilitators to raise digital literacy, emotional intelligence, critical thinking ability, and communication skills across the firm. New training curricula would require new positions to run the programs, e.g. “Director of Life-long Learning”. In this case, a job might be more akin to an education: you would leave it smarter and better-prepared than when you arrived.

The technologies coming through will also enable and require new professions and a raft of new roles might emerge across the globe in every sector as we wrestle with the ethical, legal and societal implications of machine decision making. For example, as driverless vehicles get closer to becoming a market reality, we may see the rise of the “autonomous ethicist” – specialists who attempt to work out the ethics necessary to program autonomous vehicles. This is going to be a social, moral, ethical, political, economic, and – ultimately – legal minefield. Every country, city, region, regulator, insurer, religion, civil rights group, and car manufacturer will want to contribute to the debate. The goal is to try and establish the rules and assumptions that will underpin the decision making within an autonomous vehicle as it becomes aware that it is about to have an accident.

Should a self-driving vehicle prioritize the safety of its passenger, the pedestrian who stepped in front of it, or the pregnant mother on the pavement behind them? Should it put the interests of the taxi owner over those of the driver? How will it make those choices? In making those decision, will it use facial recognition to identify individuals, and pull our tax records and other public information to work out what our net worth is to society or what our future contribution might be? How will it assess the contribution of a writer/journalist versus a baker, doctor, or actuary? What if it chooses to run down an irreplaceable hundred-year-old tree instead of a human? In a Hindu village in India, for example, running over a cow to save a passenger might be viewed as the worst possible outcome, and therefore the ethics programmed into the vehicle may prioritize the safety of the sacred animal over that of the human. Our ethicists will have to take account of all these different perspectives in constructing their guidance, and this could vary dramatically even within a country.

Q3: What are the implications for how we lead and manage tomorrow’s organizations?

Aside from jobs, bringing AI into the workplace successfully will require new workplace leadership styles. The leaders of AI-powered organizations will face unprecedented challenges which will test their people skills and emotional intelligence. “Warm” and highly relatable individuals might be in demand to offset the extent of “cold” automation within an organization. Of course, this won’t be universally true – for some, the ultimate goal is to create the DAO, and so for them the pursuit of automation and a workforce led by “robot overlords” is just a stepping stone to the employee-free business of the future. However, at present, humanity seems to be prevailing to some degree, and total digitization seems unlikely to become a genuine threat for the majority of larger global businesses in the near term.

Indeed, in a world where there’s a risk of automation, dehumanization, and commoditization proceeding hand in hand, those who put people first could find themselves better positioned to create, innovate, adapt, evolve, stand out, and outperform the market. Hence, leaders could become more important than ever, raising their own digital literacy, investing heavily in people development, and demonstrating the kind of extraordinary leadership required in an ever-evolving landscape. In many ways, the real opportunity is being ready to stand up for the longer term with this investment in people, going against a strong near-term focused, pro-AI trend that prioritizes immediate profits over humanity and future business sustainability. The emphasis on machines, processes and structures plays into – and perhaps emanates from – the dominant masculine culture in many firms. In contrast, the pursuit of a unique, distinctive, people-centered brand and culture means there could be a greater need for leading from the feminine, with an emphasis on traits such as empathy, social awareness, sensitivity, and collaborative working. Feminine might be just one word for it, but ultimately it is a perspective that puts people, relationships, and the long-term above efficiency and short-term cost savings.

Q4: What might the implications be for large and mature global companies?

Over the next decade, if things follow the “preferred future” that most nations and businesses are pursuing, the global economy could grow from about US$78Tn today to around US$120Tn. More than half of that is likely to come from businesses that have emerged recently or don’t yet exist, and over 80% will almost certainly be from products and services we don’t have today. For most, this represents a massive opportunity to innovate and evolve to ensure they maintain or attain a leadership position in their sector. The challenge is to embrace innovation at speed across the company and conduct accelerated experiments with a range of new ideas that could help generate near and long-term opportunities. So, for example, let us consider the insurance sector as a case study. Technological disruption might mean rethinking the entire approach to designing and developing policies. The speed at which markets, products, and services emerge and evolve means that increasingly we may see a shift to simple collective insurance pools, peer-to-peer, and crowdfunding models where the members of a sector ecosystem (customers and suppliers) effectively self-insure. Equally we could see per second policies where objects such as industrial machinery and domestic power tools are connected via the Internet of Things (IoT) and only insured when actually in use.

With so many sectors, products, and services emerging – from driverless cars to self-administered neural stimulation drugs and personal genetic enhancement kits – no insurance company could keep up using current approaches to identifying opportunities, assessing risks and defining appropriate coverage solutions. The responsibility will need to be handed to the sector participants to create self-service and customizable products. There will be an opportunity here for firms to become the provider of the platforms for sectors and businesses to design their own tailored insurance solutions. Rather than assume responsibility for product development, insurers could provide the software infrastructure, risk assessment models, and investment management tools and let the industry participants bear the risk.

Industries will also change, meaning a lower risk profile. Smart farms might mean fewer crop-failures, the IoT could enable smart cities with better hazard prevention, and self-driving cars should theoretically never have accidents and hence the notions of self-owning and self-insuring vehicles becomes a possibility. A range of equally dramatic developments across a range of other sectors could have potentially serious implications for insurance. Furthermore, changing lifestyles, potentially lower real-term incomes, and smart tracking technology are all driving growth of the sharing economy and scenarios where ownership is rather obsolete and most possessions are shared, not owned by one individual. This goes along with the shrinking value of owning something and instead purchasing access to it. Shared items could come insured as part of the deal, thus negating any need for buying individual policies. The risks might be borne by the users and reflected in the price. Again, the opportunity for individual firms might be to become leaders in designing customizable sharing economy policies for customers as diverse as power tool manufacturers or community ownership schemes.

The growing experience economy also creates opportunities. For the developed world and middle classes everywhere, we are at a time in history where experiences are starting to matter more than things – whilst tricky to insure, these products could take a similar form to trip insurance. Infinitely flexible policies could be designed to protect people against bad dates or wasting their time on a movie they didn’t enjoy. The payout could vary from a ticket refund through to the cost of counselling and treatment should the experience be truly traumatizing.

To enable these kinds of shifts, firms needs to ensure an effective ‘innovation architecture’ that supports a wide range of creative thought and action across all its employees globally. Key components would include ensuring leadership and management truly understand both the technologies reshaping our world and the associated mindsets that are creating new and disruptive concepts, strategies, business models, products, and services. At the local level, the freedom and capacity to conduct rapid market facing experiments is critical – as is the need to have people across the organization seeking out and connecting with emerging businesses and sectors and their respective associations. These market focused conversations are critical to understanding how current and future sectors and opportunities might evolve. The goal is to gain early access to what might become important future revenues streams. Firms might also want to consider following the path being pursued by many large organizations that are creating exponential or 10x growth and improvement programs to identify breakthrough ideas that could deliver step change gains internally and in the marketplace. The key is to let ideas blossom and see which ones create the quickest and/or biggest opportunities to take us into the future.

A final piece of the innovation mix is developing a culture of organizational foresight spread across each department and country. This involves consistently scanning the horizon to identify new and emerging societal, technological, commercial, political, and economic developments which could impact our current and future markets, products, services, and customers – or the way the firm itself operates. Embedding the importance of foresight and long-term thinking could be critical in ensuring the next 5, 50 or 100 years of existence and success.

Q5: How might emerging technologies impact the future products, services and business models of these large global players?

Over the next decade, pretty much every major business on the planet will probably evolve into a technology company almost indistinguishable from the likes of IBM or Google in its capabilities. The ability to master technologies such as AI, blockchain, the IoT, cloud computing, hyperconnectivity, and big data will simply be a ticket to the game. Success lies in the firm’s ability to leverage that portfolio of smart technologies to help unleash human potential. Let’s look at some of the possibilities – again using insurance as a case study.

Artificial intelligence – particularly when combined with other technologies – offers potentially the largest disruption. Internally the technology could transform literally every process within the business. In the marketplace, embedding AI could create wholly new concepts at the boundaries of current insurance thinking. Imagine the smart building that monitors data from tens of thousands of sensors to predict a failure somewhere in its ecosystem and calls in the appropriate inspection or repair; the smart vehicle that fines the driver for speeding and disables itself if the sensors detect alcohol on our breath; the medical monitoring device that manages drug delivery to ensure constant flow of medication. The applications are literally limitless – some may create insurance opportunities; others may open up possibilities that are currently outside our focus and comfort zone.

Sensors and their associated data are enabling the IoT which could reorient the relationship we have with our natural and physical environments and a whole range of “smart” objects. As technology becomes increasingly observant of us, and more and more human behavior is captured, stored, and analyzed, will our regulators and personal privacy preferences allow monitoring to continue along this path? There are still a lot of uncertainties in this area regarding who owns the data. Does it belong to government, like a public resource? To the people? What new insurance product and service opportunities might arise? What role might social media play in accumulating data related to insurance matters – could analysis of a person’s social media habits provide a better method of insurance risk assessment than current approaches?

Blockchain, which is the secure ledger technology underlying cryptocurrencies like Bitcoin, could help determine ownership and prevent fraud. Smart and open peer-to-peer technologies like blockchain could increase the transparency of risks and force the development of taxes to cover social risks of future technologies. In this case, is it possible that insurance companies based on inevitable surprises could emerge, highlighting and protecting against the inherent biases in technologies?

With the risk of rising economic inequality, clients may increasingly opt to share more personal data to reduce insurance rates, which might also curb their more unsafe activities and behaviors. However, in the smart city context, this may not be required as AI could take on such a large role in our lives as to advise us 24/7 – to the extent that poor decisions are automated away and eliminating the inherent biases to make risk-taking more attractive. Technology can enable more customer-centric solutions such as per day mobile phone insurance while travelling. Furthermore, the needs for insurance could be reduced through the emergence of “self-repairing assets” using 4D printed shape-shifting materials for example. Furthermore, 3D printing could allow the cost of manufactured goods to fall so much that people simply insure fewer assets. Finally, life-extension technology emerging from the marriage of Big Pharma and Silicon Valley could create a demographic, economic, and societal tidal wave. How will insurance companies respond if one future perk of wealth is the ability to buy high-tech drugs that allow lifespans of 120 or more years?

Q6: What characteristics should organizations seek to enhance or develop to ensure they have a very human future?

For organizations to navigate the decades ahead they need to see themselves as a living, breathing, constantly evolving, and very human organization – designed for and by people. This means a culture that embraces continuous innovation and experimentation on both an incremental and a dramatic scale, and a willingness to pursue exponential improvements. Such a journey requires a highly empathetic, trusting, and nurturing relationship with employees where technology is seen as a means of allowing them the time to be creative, innovative, experimental, and customer-centric. In parallel it means being seen to be supportive of those whose jobs are displaced. If one was to look at any successful brand in a decade’s time, we hope it would stand out as forward thinking, opportunity seeking, risk-aware, and far sighted in service of its customers. We would be commending its capacity to anticipate changing societal needs and risks and its willingness to adapt and evolve to deliver solutions and surprises that meet the needs of a rapidly changing reality or create new sources of delight for the customer.


Image: Alex Andreyev


Contact Us

Have a question?

Have an idea for a book or want to contribute your piece to our next project?

Interested in working for Fast Future Publishing?

E Mail:

Latest Blog

PR – Aftershocks and Opportunities 1 – Scenarios for a Post-Pandemic Future

An opportunity for fresh perspectives. While the world grapples with the current unfolding crisis, 25...Read More…

Why are US firm buying up towns? Could AI resolve geopolitical conflicts? How might Crypto transform financial markets?

New book Aftershocks and Opportunities 2: Navigating the Next Horizon bursts with provocative ideas, disruptive...Read More…

Fast Future Publishing Limited is a company incorporated in England and Wales registered number: 9484249 ©2024 Fast Future Publishing Limited