Beyond Prompts: Why Physiological Awareness is the Next Frontier for Artificial General Intelligence

Co-authored by:
David Maman, Co-Founder, CEO & CTO, Binah.ai
Michael Markzon, Co-Founder, Chief Scientist, Binah.ai
Konstantin Gedalin, PhD, Co-Founder, Chief Research Officer, Binah.ai
Naveh Tov, PhD, MD, Chief Medical Officer, Binah.ai

The current paradigm of Artificial Intelligence, dominated by Large Language Models (LLMs), operates on a model of explicit instruction. While these systems demonstrate remarkable capabilities in processing and generating information based on user-provided prompts, they remain fundamentally inert, lacking any intrinsic awareness of the user’s cognitive or emotional state. This paper argues that this “awareness gap” is the primary limiting factor preventing the transition from powerful tools to truly intelligent, symbiotic partners. We propose that the next evolutionary leap toward Artificial General Intelligence (AGI) will not be achieved through algorithmic brute force alone, but through the integration of a real-time, implicit data stream reflecting the user’s physiological state. We introduce the concept of “Biometrically-Aware AI”™ and posit that a scalable, software-only solution, capable of transforming any camera-equipped device into a passive physiological sensor, is the critical enabling technology for this paradigm shift.

1. The Age of Prompts: A Paradigm of Explicit Instruction

The last half-decade has been defined by the meteoric rise of Large Language Models and generative AI systems. Architectures like the Transformer (Vaswani et al., 2017) have enabled models such as OpenAI’s GPT series, Google’s Gemini, and Meta’s Llama to achieve unprecedented performance on a wide range of natural language tasks. Their operational modality is, however, uniform: they are slaves to the prompt.

The entire interaction loop is predicated on explicit, user-generated input. The quality of the output is directly proportional to the quality and clarity of the instruction, a reality that has given rise to the entire field of “prompt engineering” (Liu et al., 2023). These systems can be analogized to a savant assistant with encyclopedic knowledge  that lacks social or emotional intelligence. They can answer any question asked but cannot “read the room.” They are unaware if the user is focused, stressed, confused, fatigued, or frustrated. This reliance on a single, explicit channel of communication creates a profound and limiting bottleneck.

The intelligence demonstrated is reactive, not proactive or adaptive. The AI waits for a command, executes it, and returns to an inert state. This is a powerful form of information processing, but it is not a true representation of intelligence as we understand it in biological systems, which is characterized by constant, multi-modal sensory input and adaptation.

2. The Communication Gap: The Missing 93%

In human interaction, verbal communication represents only a fraction of the total information bandwidth. Research, famously pioneered by Albert Mehrabian (1971), suggests that the vast majority of meaning is conveyed through non-verbal cues, tone of voice, body language, and facial expressions. While the exact percentages are often debated and context-dependent, the core principle is undeniable: communication is a multi-layered phenomenon.

Current AI systems are deaf and blind to this implicit layer. They lack a “Theory of Mind” (ToM)the ability to attribute mental states to oneself and others (Goldman, 2012). An AI cannot infer a user’s rising cognitive load from subtle physiological cues, nor can it sense a spike in anxiety during a complex task. It is operating with an incomplete, low-fidelity model of the user.

This “awareness gap” is not a minor limitation; it is a fundamental barrier. Without the ability to perceive the user’s internal state, the Human-Machine Interface (HMI) will always remain transactional rather than relational. It prevents the development of systems that can truly collaborate, empathize, and anticipate the needs of their human counterparts.

3. The Biometric Bridge: Introducing Physiological Awareness

To bridge this awareness gap, we propose a new, implicit communication channel: real-time physiological data. We define a Biometrically-Aware AI as a system capable of passively and continuously integrating a user’s vital signs and biomarkers to build a dynamic model of their cognitive and affective state.

This “biometric bridge” provides the raw data for the missing non-verbal channel. Key indicators include:

• Heart Rate (HR): A fundamental measure of arousal and physical exertion.
• Heart Rate Variability (HRV): A powerful and medically-recognized indicator of autonomic nervous system balance. Low HRV is strongly correlated with stress, cognitive load, and fatigue (Kim et al., 2018).
• Respiration Rate: Changes in breathing patterns are directly linked to states of calm, focus, or anxiety.
• Blood Pressure: Foundational health metrics that can provide context on overall well-being.

By interpreting this data stream, an AI can move beyond mere instruction-following. It can begin to infer context and adapt its behavior accordingly. A rising heart rate coupled with falling HRV during a work task does not require a prompt; it is an implicit signal of mounting stress that an intelligent system should be able to recognize and act upon.

4. Use Cases: Transforming the Human-Machine Interface (HMI)

The integration of physiological awareness will catalyze a paradigm shift in HMI across numerous domains.

• Use Case 1: Adaptive Educational Platforms Scenario: A student is learning a difficult calculus concept using an AI-powered tutoring system on a laptop. Biometric Input: The laptop’s webcam measures the student’s mental stress level. Adaptive Response: The AI detects a sustained increase in the student’s stress level, coupled with repeated errors on a problem. Instead of presenting another similar problem, the AI intervenes: “I see this concept might be frustrating. Let’s try a different approach,” and switches to a more foundational video explanation or a simpler example.
• Use Case 2: Empathetic Digital Health Companions Scenario: A digital wellness companion app on a smartphone aims to help a user manage anxiety. Biometric Input: The phone’s front-facing camera takes periodic, passive stress-level, and other vitals, readings throughout the day using Binah SDK performing measurements while the use uses the smartphone for daily activities. Adaptive Response: The AI notices a pattern of elevated stress every workday around 2 PM. It proactively sends a notification: “I’ve noticed this time of day can be stressful. Would you like to try a 2-minute guided breathing exercise?” This transforms the app from a reactive tool to a proactive, caring partner.
• Use Case 3: Proactive AI Health Monitoring Scenario: An AI companion integrated into a user’s personal devices (phone, tablet) is tasked with long-term health and wellness oversight. Biometric Input: Using Binah.ai‘s SDK, the AI prompts for and tracks weekly measurements of bloodless biomarkers like Hemoglobin, Hemoglobin A1C, and Cholesterol, in addition to daily stress and vitals. Adaptive Response: The system detects a concerning multi-week downward trend in Hemoglobin levels while noting a simultaneous increase in reported fatigue. The AI cross-references these data points and intervenes: “I’ve noticed some trends in your recent wellness checks that might be worth discussing with a professional. Would you like me to find and schedule an appointment with your primary care physician?”
• Use Case 4: Dynamic Tone Adaptation in Communication Scenario: A user is interacting with a work-focused AI assistant (e.g., in Slack or email) to manage a tight deadline. Biometric Input: The device camera passively measures the user’s cognitive load via HRV. Adaptive Response: When the user’s cognitive load is high, the AI’s responses become more concise and direct, providing bullet-pointed summaries and clear action items. When the user is relaxed, the AI adopts a more conversational and expansive tone. This fluid adaptation ensures the AI’s communication style always matches the user’s capacity and need.
• Use Case 5: Personalized AI Nutrition Guide Scenario: A nutrition and meal-planning AI helps a user optimize their diet for their health goals. Biometric Input: The AI uses real-time physiological and bloodless biomarkers data (stress levels, HR, respiration, Hemoglobin, Hemoglobin A1C, Cholesterol) to understand the user’s current state. Adaptive Response: After a stressful meeting, the AI detects elevated stress levels. Instead of suggesting the user’s standard pre-planned meal, it suggests an alternative rich in magnesium and theanine, explaining, “Your stress levels seem a bit high. A meal with these ingredients could help promote calmness. Here is a quick recipe.”
• Use Case 6: Cognitive Load Management in Augmented Reality (AR) Scenario: A surgeon uses Meta’s Reality Labs AR glasses during a complex procedure, with anatomical overlays and vital patient data displayed in her field of view. Biometric Input: The headset’s camera, using Binah SDK, passively monitors her HRV and respiration rate. Adaptive Response: As the procedure reaches a critical stage, the system detects a significant drop in her HRV, indicating high cognitive load. The AI autonomously simplifies the display, fading non-critical information and highlighting only the most essential data points to reduce distraction and support focus.

5. The Scalability Challenge and The Software-Only Solution

The vision of a Biometrically-Aware AI has historically been hampered by a critical obstacle: hardware. Traditional methods for capturing physiological data rely on dedicated, contact-based hardware such as ECG chests straps, pulse oximeters, and smartwatches. While valuable, these solutions suffer from three fundamental limitations for mass adoption:

1. High Friction: Devices like smartwatches or wearables require users to purchase, wear, and charge a separate device, which can hinder consistent usage.
2. Active Participation: The user must consciously decide to put on the device for a measurement to occur.
3. Lack of Ubiquity: They are not integrated into the primary computing devices (laptops, desktops, AR/VR systems) where advanced HMI is most relevant.

The only way to achieve ubiquitous physiological awareness is through a frictionless, software-only solution.

This is the strategic importance of Binah.ai‘s Health Data Platform. By utilizing advanced AI and signal processing, our technology extracts a rich set of vital signs and biomarkers from a video stream captured by any standard RGB camera (rPPG technology).

Delivered as a Software Development Kit (SDK), Binah.ai‘s technology can be embedded directly into the operating system or application layer of any device. This turns billions of existing smartphones, tablets, and AR/VR headsets into powerful, passive biometric sensors without any additional hardware. It is the only solution that solves the scalability challenge, offering a frictionless path to making every human-machine interaction biometrically aware.

6. Conclusion: Towards a Symbiotic Intelligence

The pursuit of Artificial General Intelligence is at a crossroads. We can continue down the path of building ever-larger models that are faster and more accurate at processing explicit commands, or we can embark on a new path toward creating systems that truly understand their human partners.

This paper contends that the latter path is the only one that leads to true intelligence. The integration of a passive, real-time physiological data stream is not an incremental improvement; it is a categorical leap forward. It endows machines with a semblance of empathy and situational awareness, transforming them from tools into collaborators. By bridging the awareness gap, Biometrically-Aware AI promises a future of symbiotic intelligence, where technology doesn’t just respond to us but adapts with us, creating a safer, more intuitive, and profoundly more human technological landscape.

7. References

1. Goldman, A. I. (2012). Theory of mind. In The Oxford handbook of philosophy of cognitive science.
2. Kim, H. G., Cheon, E. J., Bai, D. S., Lee, Y. H., & Koo, B. H. (2018). Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature. Psychiatry Investigation, 15(3), 235–245.
3. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023). Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Computing Surveys, 55(9), 1–35.
4. Mehrabian, A. (1971). Silent Messages. Wadsworth, Belmont, California.
5. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is All You Need. Advances in Neural Information Processing Systems 30.