By Mandeep Chauhan, Founder & CTO, Awareye
Industrial firms have an enormous opportunity to cut losses and boost efficiency by tackling downtime head-on. A recent report, ‘The True Cost of Downtime’, found that large manufacturing firms experience an average of 323 hours of downtime annually, costing a staggering $532,000 per hour. That adds up to $172 million per plant each year. For Fortune Global 500 manufacturing and industrial firms, this translates to 3.3 million hours of lost productivity, amounting to $864 billion, nearly 8% of their annual revenues.
One of the biggest culprits? Safety-related downtime. In industries where humans and machines work side by side—whether in automotive plants, aerospace, or oil and gas—routine maintenance, safety protocols, and compliance checks are necessary but often eat into operational hours. While achieving 100% uptime is nearly impossible, unplanned downtime is the real nightmare, bleeding revenue by the minute.
This is where AI-driven safety solutions come into play. The rise of multimodal AI, where vision, sensors, and audio data combine to create real-time situational awareness, offers a game-changing approach to minimizing downtime and enhancing workplace safety. In this article, I’ll explore how AI can help improve our collaboration with machines in industrial settings, reduce downtime, and improve safety.
Multimodal AI Systems for Human Machine Collaboration
Let me first explain briefly what ‘multimodal AI’ is. Multimodal AI is an advanced form of AI that can process and interpret multiple types of data—such as text, images, video, audio, and sensor inputs—simultaneously. Unlike traditional AI models that focus on a single data type, multimodal AI combines different streams of information to create a richer, more contextual understanding of a situation. A great use case for multimodal AI is in industrial settings, where it can help with predictive maintenance and workplace safety.
You can think of these systems as AI that can hear, see and feel, with cameras, microphones, LiDAR and IoT sensors providing feeds of information. Our goal, when engineering such systems, would be to combine and correlate the data streams and derive comprehensive situational awareness.
As I mentioned before, these systems use an ensemble of AI models. When using RTSP video streams from cameras in industrial settings, we use trained versions of models like the YOLO series, or FasterRCNN. LiDAR systems use frameworks like the PCL (Point Cloud Library), and can be integrated with Robot Operating System (ROS). With audio data, we use Autoencoder Neural Networks and Anomalous Sound Detection (ASD) techniques.
This ensemble of AI models, deployed on-prem in industrial settings, can simultaneously work on the input streams to create comprehensive sensory awareness. Such a system is frequently integrated with PCBs or other industrial components to trigger instant alerts.
In robotics-heavy environments, they significantly reduce risk. They can be used to identify workers without proper personal protective equipment (PPE) or detect proximity between humans and machines, sending alerts to prevent accidents. They can be integrated with industrial control processes through standard protocols such as Modbus and EtherNet/IP.
Multimodal AI systems also help improve safety by predicting machine failures before they happen, reducing unexpected breakdowns and preventing equipment-related injuries. For instance, Anomalous Sound Detection (ASD) can be used on audio streams to recognize unusual noises indicating machinery failure or unsafe conditions, among other use cases.
Role of AI in EHS Compliance
Multimodal AI can help industries adhere to Environmental Health and Safety (EHS) compliance, which involves a broad set of safety protocols, including OSHA (Occupational Safety and Health Administration) standards in the USA, or The Factories Act of 1948 and the National Policy on Safety, Health, and Environment at Workplace in India. The guidelines include protection against falls on factory floors, preventing exposure to harmful chemicals, providing safety equipment, training and assistance to employees, and using signs, labels, color codes, or posters to warn employees of potential hazards.
How can multimodal AI help here? Let’s look at some examples.
Multimodal AI systems like IoT sensors and LiDAR can monitor human-machine proximity to prevent collisions or accidents. AI can analyze sensor data (temperature, pressure, vibrations) to predict equipment failures. AI-driven fire, gas leak, or chemical spill detection can trigger immediate response actions. AI can also automate EHS audits by monitoring the safety guidelines for compliance in real time.
Ultimately, compliance is more than a legal requirement; it’s a moral and financial responsibility. By adhering to these safety standards, companies can avoid costly fines and legal issues. Additionally, they can build a culture of responsibility and care, boost reputation, and instill confidence in partners and clients.
Stepping into Industry 5.0
The use of Multimodal AI in industrial settings is part of a greater revolution termed Industry 5.0. Where Industry 4.0 was about automation and data, Industry 5.0 focuses on the collaboration between humans and advanced technologies like artificial intelligence (AI), robotics, and the Internet of Things (IoT). Its ultimate goal is to create more personalized, efficient, safe and sustainable manufacturing processes.
If numbers are to go by, we are already in this transitionary phase. The global Industry 5.0 market was valued at approximately $65.8 billion in 2024 and is projected to reach around $255.7 billion by 2029, a CAGR of 31.2%. If industries are to remain competitive, it makes sense to invest in multimodal solutions now before everyone gets on the bandwagon.
How can you do this? Start with a POC (Proof of Concept), that is, pilot a small-scale multimodal AI project in a high-impact area (e.g., PPE detection, predictive maintenance, or anomaly detection). You can use readily available AI models (YOLO, Faster R-CNN, Autoencoders) to test feasibility before scaling. Finally, integrate with your existing CCTV, IoT sensors, and industrial automation systems to collect real-time data.
Staying Ahead
The future is a workplace where technology actively protects human workers while optimizing operational efficiency. It is a future that focuses on intelligent, responsive, and human-centric AI solutions. So the question is no longer if AI should be integrated in industrial settings, but how quickly businesses can implement multimodal AI technologies to stay ahead in the competitive landscape of Industry 5.0.