251104P - ETHICAL CHALLENGES OF AI IN HEALTH CARE? 2

2nd International Nursing Congress Ministry of Defense Health Services held on 2-4 November 2025 at Crown Plaza RDC Riyadh, KSA by Prof. OMAR HASAN KASULE MB ChB (MUK), MPH (Harvard), DrPH (Harvard), Professor of Epidemiology and Bioethics, Chairman of the Human and Research Ethics Committees at King Abdullah bin Abdulaziz University Hospital in Riyadh

MAJOR CHALLENGES OF AI IN HEALTHCARE:

• Lack of a moral vision for the future.
• Rapid changes are leaving many health care workers confused. Generative AI is a new data development. New logical and analytic systems are coming up every day; they may change the risk/benefit balance.
• The possibility of major mistakes from biased databases, wrong algorithms, or analysis is a result of the wrong cultural context.
• Loss of human control; humans controlled by machines.
• Ethico-legal liabilities for errors by AI.

FUTURE VISION:

• What is the long-term moral vision? Who knows where we are and where we are going?
• The concept of purpose (gha’iyyat). Everything in the universe has a purpose. 
• What is the future purpose of AI? Do we know only the purpose of today? Have we thought about tomorrow?
• Besides jobs, does AI threaten human moral superiority?

HUMAN MORAL SUPERIORITY:

Allah gave humans superiority over all other creations in the cosmos through 5 enablers:

1.  Istikhlaf (Vicegerency)
2.  Haml al amanat (bearing responsibility)
3.  Taskhir (control)
4.  Isti’mar (civilization)
5.  Akhlaq (morality)

ISTIKHLAF (VICEGERENCY):

• Istikhlaf (AlBaqara: 30). 

• وَإِذْ قَالَ رَبُّكَ لِلْمَلَائِكَةِ إِنِّي جَاعِلٌ فِي الْأَرْضِ خَلِيفَةً ۖ  قَالُوا أَتَجْعَلُ فِيهَا مَن يُفْسِدُ فِيهَا وَيَسْفِكُ الدِّمَاءَ وَنَحْنُ نُسَبِّحُ بِحَمْدِكَ وَنُقَدِّسُ لَكَ ۖ قَالَ إِنِّي أَعْلَمُ مَا لَا تَعْلَمُونَ. [البقرة: ٣٠]   

HAML AL AMANAT (RESPONSIBILITY):

• Haml al amanat (AlAhzab: 72). 

• إِِنَّا عَرَضْنَا الأَمَانَةَ عَلَى السَّمَوَاتِ وَالأَرْضِ وَالْجِبَالِ فَأَبَيْنَ أَنْ يَحْمِلْنَهَا وَأَشْفَقْنَ مِنْهَا وَحَمَلَهَا الإِنْسَانُ إِنَّهُ كَانَ ظَلُومًا جَهُولًا. [الأحزاب: ٧٢]   

TASKHIIR (CONTROL):

• Taskhiir (AlJathiyah: 13). 

• وَسَخَّرَ لَكُم مَّا فِي السَّمَاوَاتِ وَمَا فِي الْأَرْضِ جَمِيعًا مِّنْهُ ۚ إِنَّ فِي ذَٰلِكَ لَآيَاتٍ لِّقَوْمٍ يَتَفَكَّرُونَ. [الجاثية: ١٣]

• Under the Qur’anic paradigm of taskhiir (14:32-33, 31:20, 45:13), the human was endowed with special intellectual power in the form of the largest cerebral cortex area/body volume to control the rest of creation (living and non-living).
• In addition, the human was given a flexible figure that enables him/her to make tools such as computers, which must remain under human control according to the doctrine of taskhiir.

ISTI’MAR (CIVILIZATION):

• Building civilization, isti’mar (Hud: 61). 

• وَإِلَىٰ ثَمُودَ أَخَاهُمْ صَٰلِحًا ۚ  قَالَ يَٰقَوْمِ ٱعْبُدُواْ ٱللَّهَ مَا لَكُم مِّنْ إِلَٰهٍ غَيْرُهُۥ ۖ هُوَ أَنشَأَكُم مِّنَ ٱلْأَرْضِ وَٱسْتَعْمَرَكُمْ فِيهَا فَٱسْتَغْفِرُوهُ ثُمَّ تُوبُوٓاْ إِلَيْهِ ۚ  إِنَّ رَبِّى قَرِيبٌ مُّجِيبٌ. (هود: ٦١)

• Because of taskhiir the humans can build a civilization which is good or bad.

AKHLAQ (MORALITY):

• Humans have an innate moral judgment to distinguish the right from the wrong highway, najdain (AlBalad: 10). 

• وَهَدَيْنَاهُ النَّجْدَيْنِ. [البلد: ١٠]

CONCEPT  OF  TASKHIIR (CONTROL):

• وَسَخَّرَ لَكُم مَّا فِي السَّمَاوَاتِ وَمَا فِي الْأَرْضِ جَمِيعًا مِّنْهُ ۚ إِنَّ فِي ذَٰلِكَ لَآيَاتٍ لِّقَوْمٍ يَتَفَكَّرُونَ . الجاثية: 13

• Humans are in charge because of the largest cerebral surface area to body weight and an apposable thumb, allowing them to make tools.
• Humans control animals and plants.
• Human manipulates the environment.
• Human makes tools to extend their power of taskhiir.
• Machines and all tools are under human control.

HUMAN-MACHINE INTERACTION:

• Before recent developments of autonomous artificial intelligence, the computer was a decision support system under the programmer who could predict all its decisions.
• In the modern AI era, the computer still functions as a decision support tool but has some ability to learn on its own, to think, and to decide solutions to solve problems creatively, such as disease diagnosis, disease drug treatment, and surgical procedures, as well as control other computers and machines producing goods and services.
• The human may not predict or envisage what the machine will do. Thus is a threat to human control (taskhiir).

HUMAN-MACHINE  INTERACTION BEFORE THE ERA OF AI:

• Before AI, the human programmer was in charge; the computer could not work outside what he envisioned, and could not come up with unpredictable things.
• The computer was an extension of human faculties and a mere tool.
• The advantage of the computer was speed, processing a large amount of data, and a huge memory.
• Human control was never challenged in the past as it is today by AI.

HUMAN-MACHINE  INTERACTION IN THE ERA OF AI:

• The computer now has some ability to think and solve some problems creatively.
• The computer can control itself and control other computers.
• The computer can make products and machines; the machines make more products.
• Has the human lost control?
• What are the ethical implications of loss of human control?

OLD ROLE OF THE COMPUTER IN MEDICINE: Decision support

• The computer has relieved the human physician of routine mechanical thinking processes. It had the advantage of speed and consistency (fewer systematic errors).
• A computer was an aid to diagnosis by processing a lot of data quickly to narrow down diagnostic choices, but in the end, it was the human who decided.
• The computer summarized information for the human physician to make a decision.
• The computer saved the human from routine, mechanistic brain operations that do not involve higher thinking.

NEW ROLE OF THE COMPUTER:

• Deep learning or machine learning technologies have enabled AI to enter widely into medicine. These involve some form of thought and creativity.
• AI is used in disease diagnosis, triage or screening, risk analysis, and surgical operations. AI is used in the early detection of skin cancer in community settings.
• Use of AI in image analysis for breast cancer screening.
• AI is used in cardiology.
• AI used to diagnose secondary hypertension.

USE OF AI IN IMAGING:

• Labelling thyroid nodules.
• Diagnosing lung cancer from chest CT images.
• Diagnosis of liver tumors.

USE OF AI TO CONTROL INSULIN PUMPS:

• Decision-making on how much insulin to inject is based on monitoring blood sugar.
• A comparison was made between a physician and AI, and no significant disagreement was found.

IMAGE ON AN AI-CONTROLLED INSULIN PUMP:

USE AI IN SURGERY:

• AI controls robotics that carry out surgical procedures, but always under the control/supervision of the surgeon.
• Machine learning involves computers deriving information from images and videos, reaching some conclusions, and then taking action.
• Surgical robots such as the da Vinci Surgical System allow remote surgery.
• Algorithms of AI: reliable?

ETHICAL QUESTION OF LIABILITY:

• Who is responsible for errors of the computer: the programmer, the manufacturer, the attending physician, or the hospital?
• Mistakes in diagnosis.
• Mistakes in procedures.
• Mistakes in prognosis.
• Criminal manipulation.

THE ISSUE OF CONTROL AS IT RELATES TO TASKHIR:

• The debate is still open whether humans have lost control or, worse, whether humans are under machine control.
• Can the human envisage or predict all that the machine may decide and do?
• Can the human be completely surprised by the conclusions and decisions of the machine?

THE PARALLEL OF A TRAINED HUNTING DOG (SAHIH MUSLIM 1929):

• ثم سأل عَدِيٌّ رضِيَ اللهُ عنه رسولَ الله صلَّى اللهُ عليه وسلَّم عَنِ الصَّيْدِ بالكَلْبِ المُدَرَّبِ والمُعَلَّمِ، فبيَّن له النَّبيُّ صلَّى اللهُ عليه وسلَّم: إذا أرسلْتَهُ وذكَرْتَ اسمَ اللهِ عليه عند إرسالِه، فأَمْسَكَ صَيدًا دونَ أنْ يأكُلَ منه؛ فهو حِلٌّ أكْلُهُ. فسأل عَدِيٌّ رضِيَ اللهُ عنه عن الكَلْبِ إذا أكَلَ مِنَ الصَّيدِ قَبْلَ أنْ يَأتيَ به، فنهاه النَّبيُّ صلَّى اللهُ عليه وسلَّم عن أكلِ الصَّيدِ في هذه الحالةِ؛ لأنَّ الكَلْبَ لم يَحْبِسْهُ ويُوقِعْهُ لك، وإنَّما صاده وحبَسه لنفسِه؛ ليَأكُلَه هو، ولا يَحِلُّ لك. 

• In this case, the human knows what the dog may do or not do: it may hold the animal until the hunter arrives, it may bite at the meat or let another dog catch the animal, etc., but none of these will surprise the human hunter.
• The hunter has in his intellectual repertoire all that the dog may do.
• The hunter knows what is right and what is wrong.
• The integrity of taskhiir is in no doubt. 
• Generative AI is a new challenge. The machine generates information on which it depends.

ISSUES OF LIABILITY IN THE USE OF AI IN MEDICINE:

• Assuming the human has lost control raises the legal and ethical question of who is liable (machine vs human) in cases of mistakes by AI that result in residual harm to the patient.
• Liability for mistakes in the diagnosis, the procedures, and the prognostication.
• Liability also arises in cases of potential criminal manipulation of AI.
• In case of a medical error, who is professionally accountable and liable: the programmer, the seller of the software, the owner of the software, or the physician in closest contact with the patient? 
• If all are liable, how do we portion out the vicarious liability?

LOSS OF TASKHIIR:

• If the human loses control and we pray that this does not happen, then human ethical superiority and the concept of taskhiir are jeopardized.
• The jeopardy is most severe if it involves moral violations because the machine has no moral judgement, only human has.
• Assuming the human has lost the endowment of taskhiir and is under AI control, we must discuss legislation that will modify or, if necessary, stop unwanted AI developments.

LOSS OF VISION:

• Developments in IA are driven by curiosity (scientific and technological) as well as the competitive profit motive to produce newer and more expensive technology.
• Lack of a forward vision (gha’iyyat) may lead to human disaster – where are we going and why are we going there?
• With loss of vision, we experience other losses:
• Loss of khilafat.
• Loss of amanat.
• Loss of the moral compass.

MORAL CHALLENGES INSOLUBLE BY AI:

• Comparative ethics that require a social and religious reference. If you have one kidney and both mother’s and father’s needs for it are equal.
• Contextual issues requiring cultural intelligence: AI formulating a questionnaire about previous marriage and the number of children.

THE WAY FORWARD - 1:

• These questions need discussion to reach generally acceptable legal rulings by jurists (fuqaha) in consultation with physicians and information technology experts.
• Fiqh includes revealed knowledge (‘ilm naqli) and human knowledge (‘ilm ‘aqli) and is a combination of positive and moral law.
• The paradigm of integration of knowledge, takaamul al ma’arifat, is a judicious balance of application of the two sources of knowledge to solving practical problems.
• The bird’s eye view from the maqasid al shari’at.

THE WAY FORWARD - 2:

• There should not be a rush to judge.
• Wait to see actual consequences; benefit from others’ experiments (raises a moral question).
• Conservative and prudent.

THE WAY FORWARD - 3:

• Balance the good and the bad.
• If good = bad then follow the legal maxims, qawa’id fiqhiyyat.

• درء المفاسد أولى من جلب المصالح.
• فإذا تعارضت مفسدة ومصلحة، فدفع المفسدة مقدم في الغالب، إلا أن تكون المفسدة مغلوبة؛ وذلك لأن اعتناء الشرع بترك المنهيات أشد من اعتنائه بفعل المأمورات، لما يترتب على المناهي من الضرر المنافي لحكمة الشارع في النهي.