In recent years, sporadic medical disagreements have arisen. Inadequate communication between the medical team and patients, together with their families, accounts for around one-third of medical mishaps, according to data from the Taiwan Patient Safety Net. Misunderstandings and distrust will ensue when a substantial information void exists between the attending physician and the anxiously expecting family members.
During the second "2025 AI Wave: Taiwan Generative AI Applications Hackathon," the "Hero Song" team introduced a novel solution that integrates edge computing and AI technology, enhancing doctor-patient communication and securing victory in Advantech's "Intelligent Edge-Cloud Control" category.
The members of the Hero Song team are from the Computer Science department at National Tsing Hua University in Taiwan. They discovered that the medical profession is confronted with two significant challenges following a comprehensive analysis. The initial issue is that there is a significant information divide between family members and physicians. The complete diagnosis and treatment instructions are not comprehended by family members, which leads to an increase in medical safety risks. The second reason is that family members frequently experience anxiety and the doctor-patient relationship is fraught as a result of their inability to comprehend the condition and the doctor's advice in real time. In order to resolve the aforementioned issues, the team developed "MediCAM," a cutting-edge medical communication system that integrates generative AI technology with Advantech's industrial AI camera (ICAM) and Amazon Web Services (AWS) cloud services.
MediCAM comprises four principal functions: the first is "real-time image streaming," enabling family members to remotely observe patients' current conditions; the second is "automatic summary of medical advice," wherein the system records and converts doctors' instructions into comprehensible text; the third is "instant question and answer regarding patients' conditions," allowing family members to inquire about patients' statuses using natural language and receive informed responses based on actual medical data; the fourth is "AI daily patient status report," which autonomously generates a concise report on patients' conditions daily to assist family members in monitoring recovery progress. Not only can this system eradicate the information asymmetry gap, but it can also establish a foundation of mutual trust between doctors and patients, thereby significantly reducing medical risks caused by poor communication.
The technical architecture of MediCAM is thoroughly designed, integrating cloud AI and edge computing.
The team installed Advantech ICAM AI camera edge devices adjacent to patients' beds to collect on-site visual and auditory data. The system utilizes the Whisper model for voice-to-text and Amazon SageMaker, a machine learning service, to transcribe the doctor's instructions. Additionally, Yolo and FaceNet technologies are employed to verify the doctor's identity and perform facial recognition to ensure the legitimacy of the information source.
The team employed Amazon Nova Pro on Amazon Bedrock, a fully managed service that makes high-performing foundation models, to process medical order summaries and enable intelligent question-and-answer capabilities. They used a retrieval-augmented generation (RAG) architecture to ensure the precision and dependability of the results. All data is encrypted and transmitted securely to protect patient confidentiality. AWS provides reliable support for streaming, access, and security control, enabling the consistent processing of large quantities of image, audio, and text data.
MediCAM, which is both practical and inventive, was the standout technology proposition in the Advantech technology proposition category for the "Hero Song" team.
The jury stated that this system possesses numerous advantages: the high level of immediacy enables family members to comprehend the patient's condition promptly; the information is transparent and objective, while the automatic transcription and summarization of medical orders prevent information omissions or misinterpretations; the intuitive interactive experience permits users to obtain answers by merely posing questions in everyday language, significantly reducing the barrier to understanding. The solution utilizes AWS cloud services and can be rapidly scaled to multiple hospital campuses. This technology not only significantly diminishes the likelihood of ineffective communication between doctors and their patients, but also facilitates the integration of AI applications into Taiwan's healthcare system.
Article edited by Sherri Wang
