LG Telehealth Platform

LG's first market-ready telehealth product: virtual rounding, AI-assisted patient safety monitoring and an AI scribe, launched with Amwell.

Company: LG Electronics
Year: 2021 — 2023
Role: Project Leader · product definition, architecture, APIs, hands-on development, partner management
Team: Emerging Technologies Department, Silicon Valley Lab
Scale: Two cross-functional teams, 18 people across 4 states and 2 countries
Location: Santa Clara, CA

Technology

WebRTC
React
TypeScript
Cloud call and device management
Edge AI on NPU
Skeletal tracking
LLM summarization
Custom hardware (LG SmartCam Pro)

01The opportunity

Hospitals were under pressure to care for more patients with fewer people on the floor. Remote consultations and remote patient safety monitoring were proven ideas, but most systems bolted consumer video tools onto hospital workflows, with fragile hardware, poor accessibility for patients and little intelligence.

LG had the connected-device expertise, the hospital TV footprint and, through LG NOVA's Digital Health group, a go-to-market partner in Amwell. The question was whether we could build a complete, reliable telehealth platform for both patients and providers, and make it good enough to ship.

02The idea

A single platform that covers the two workflows hospitals actually run: Virtual Rounding, where providers, specialists, family members and translators join a patient's call, and eSitting, where patient safety monitoring (PSM) staff watch up to sixteen patients at once and intervene remotely.

Patients should need almost nothing to use it. Calls are accepted by voice, or auto-connected for patients who cannot speak. There is no remote to lose or break. Providers, in contrast, get full control: participant management, multi-camera switching, PTZ and ePTZ camera control, screen and image sharing, alarms and pre-recorded messages.

AI runs where it is useful and where the data lives: on the edge device in the room, for patient tracking and Smart Rails, and in the cloud, for AI Scribe call transcription and summaries.

03My role

I led the platform from proof of concept to LG's first market-ready telehealth product. I ran two cross-functional teams, 18 people in total across four states and two countries, through the delivery of a system with more than forty features.

I owned the product definition, informed by user interviews and visits to hospitals and eSitter centers, designed the platform's APIs for scalability, and contributed directly to the codebase in React and TypeScript.

I managed the relationships that made the product possible: the go-to-market partnership with Amwell, LG's CTO and CSO divisions at headquarters in Korea, and the LG NOVA Digital Health team in Santa Clara.

04How it worked

The platform combines LG's custom patient-side hardware with a cloud call management system (CMS) and device management system (DMS). Video and audio run over WebRTC. Providers and PSM staff work from browser-based consoles; patients see the call on the in-room LG TV.

The patient-side device, the LG SmartCam Pro, carries dual RGB and infrared cameras, network and TV connectivity, on-board compute and an NPU for real-time AI on the edge. It works with additional healthcare cameras such as a 30x AVer camera, which is precise enough to observe pupil movement remotely.

Smart Rails: The platform's first AI feature. A PSM draws a six-point polygon around a bed or an area of the room; the system tracks the patient's skeleton and alerts when they cross it. Inside-out triggers catch patients leaving the bed, outside-in triggers protect restricted areas. Three sensitivity levels decide whether limbs, torso or head count. A 2023 update recognizes the bed automatically and places the boundary itself, day or night, using the IR camera.
eSitter console: A dynamic grid of up to sixteen live feeds with per-patient notes, focus mode, interaction mode for two-way audio, camera selection, PTZ control, alarms, a sound monitoring bar, blur mode for privacy during procedures, a listen-in feature, and pre-recorded messages in five languages.
AI Scribe: When enabled, calls are transcribed in real time and an LLM produces a summary with discussion points and action items. Providers can dictate additional notes after the call and have them folded into the summary.

05Prototype / product

eSitter console showing a grid of twelve live patient video feeds with a patient detail panel on the left — Fig. 01ProductionThe eSitter console. Patient safety monitoring staff observe up to sixteen patients at once. The left panel holds editable notes and non-editable clinical details for the selected patient; tools include interaction mode, camera selection and control, and alarms for on-floor staff.

Patient-side view of a virtual rounding call on the in-room TV — Fig. 02ProductionVirtual Rounding from the patient's side. Minimal controls: patients accept or reject calls by voice, or the system auto-connects for patients who cannot speak.

Provider-side virtual rounding call with participant list, camera controls and a shared X-ray image — Fig. 03ProductionThe provider side. Participants can include other providers, family members or translators. Here the provider shares an X-ray with everyone on the call.

eSitter console with one patient enlarged in focus mode and interaction mode active — Fig. 04ProductionFocus mode enlarges one feed so the PSM can watch closely, for example when a patient pulls on IV lines. This patient is also in interaction mode, with two-way audio open.

eSitter console with an alarm triggered in one room while interacting with another patient — Fig. 05ProductionLate-2022 build. The PSM is in interaction mode with one patient (green boundary) while triggering an alarm in another room. Also visible: the sound monitoring bar, pre-recorded messages, blur mode and listen-in.

Smart Rails setup: a six-point polygon drawn around a hospital bed with alert sensitivity settings — Fig. 06ProductionSmart Rails setup. The PSM defines a polygon with adjustable vertices around the bed or a restricted area; the system tracks the patient's skeleton and alerts on boundary crossings. Sensitivity settings decide which body parts count.

Automatic bed recognition placing Smart Rails boundaries around a hospital bed — Fig. 07ProductionEarly 2023: the system recognizes the bed and places the boundary automatically, even when only part of the bed is visible from a suboptimal camera position.

Smart Rails running in night vision with the infrared camera — Fig. 08ProductionThe same feature in night vision, running on the IR sensor as well as the RGB camera. Every aspect of Smart Rails is covered by an extensive patent application.

Provider dictating additional notes after a telehealth call — Fig. 10PrototypeAI Scribe during development: the provider dictates post-call action items.

AI Scribe recording indicator during a call — Fig. 11PrototypeDictated notes are folded into the summary.

Transcript and LLM-generated summary of a telehealth call — Fig. 12PrototypeThe transcript and the LLM summary with key discussion points and action items. Early UI, limited text formatting.

Fig. 13ShippedThe Virtual Rounding call system demonstrated by Atul Singh, General Manager, Digital Health at LG NOVA, in an interview with Healthcare IT Today.

LG SmartCam Pro patient-side device, front view — Fig. 14ShippedLG SmartCam Pro, the patient-side hardware: dual RGB and IR cameras, network and TV connectivity, on-board compute and an NPU for real-time AI on the edge.

LG SmartCam Pro patient-side device, rear view with ports — Fig. 15ShippedPorts for network and TV connectivity. Specification sheet linked below.

06Outcome

The platform shipped as a commercial product in 2022, a fully integrated end-to-end telehealth solution for hospitals launched in collaboration with Amwell. It produced two patent applications for novel aspects of the system, including Smart Rails, and drew coverage from healthcare press including HIT Consultant, Fierce Healthcare and Healthcare IT Today.

It is also the direct line to my current work: building AI-enabled digital health products as Head of Engineering & Product at ReliefAI Health, an LG NOVA-backed venture.

Commercial launch with Amwell, 2022
40+ features across Virtual Rounding and eSitting
Edge AI patient safety monitoring (Smart Rails)
Two patent applications

07Patents, press, links

Press

Links