Johnson & Johnson joins hands with Nvidia to develop AI surgery system

On March 18, Johnson & Johnson Medical Technology announced a partnership agreement with Nvidia to jointly develop new artificial intelligence applications

Under the terms of the agreement, J&J will gain access to Nvidia's IGX edge computing platform and Holoscan edge AI platform. It aims to drive surgical improvement by leveraging Nvidia's computing platform and artificial intelligence technology to support surgical software and equipment for real-time data analysis in the operating room.

At the same time, the two sides also plan to apply AI technology in the field of medical image analysis and disease diagnosis. This will provide doctors with more comprehensive medical information to help them make accurate diagnosis and treatment decisions more quickly.

The cooperation will also cover medical education and training. By leveraging Nvidia's virtual reality and augmented reality technologies, medical students and physicians can conduct more realistic and effective training that elevates their skill level and professionalism.

Shan Jegatheeswaran, global Head of digital at Johnson & Johnson Medical Technologies, said: "One of the major challenges to scaling AI in surgery is the closed nature of the surgical technology. Bringing advanced edge computing hardware and software into the operating room enables innovative scalability and new AI-driven solutions for clinical decision making, education and training, and collaboration, with the ultimate goal of advancing patient care."

01 Medical Edge AI NVIDIA IGX

NVIDIA IGX is an advanced hardware and software solution designed by Nvidia for edge computing applications that integrates Nvidia's latest AI technologies, including deep learning algorithms and neural network models, to make devices intelligent. It aims to provide powerful artificial intelligence (AI) capabilities and real-time data processing for a variety of fields, including healthcare.

For edge AI use cases in the healthcare industry, IGX provides secure, low-latency AI reasoning to meet clinical needs for immediate insight into a range of instruments and sensors in medical procedures, such as robot-assisted surgery, patient monitoring systems, and more.

In addition, the IGX platform supports NVIDIA Clara Holoscan, a real-time AI computing platform for medical devices, enabling medical device developers to connect the edge, on-premises data center and cloud services, enabling rapid development and deployment of new devices, and bringing AI applications directly into the operating room. More than 300 million surgeries worldwide are performed here each year.

The Holoscan platform combines hardware systems for low-latency sensors and network connectivity, optimized libraries for data processing and artificial intelligence, and core microservices to run streaming, imaging, and other applications, from embedded to edge to cloud.

It can make real-time data processing become a reality through high throughput data transmission and processing. For example, real-time analysis of pathological data. The amount of pathological image data is extremely large, even up to several terabytes. Real-time analysis of pathological data is virtually impossible. Using the Clara Holoscan platform, the researchers took the analysis time of pathological images of cancer cells from two days to real time, and for the first time, they found a single cancer cell dividing into three cancer cells in real time.

Updated 25 medical microservices

Notably, on the same day that the partnership agreement with Johnson & Johnson was signed. Nvidia also introduced a new AI inference server, Nvidia NIM, a new software platform designed to simplify the deployment of custom and pre-trained AI models in production environments. With NIM, users can experience a variety of AI models and applications while retaining full control over intellectual property.

Based on these microservices, healthcare companies can screen trillions of drug compounds, collect more patient data to aid in early disease detection, or implement smarter digital assistants, among other things.

02 Strong generative force artificial intelligence

As the world's largest and most diversified healthcare company, Johnson & Johnson has long ranked as the world's leading medical device leader, but it was later surpassed by Medtronic. In recent years, the company has been making efforts in the field of surgical robots, trying to transform itself into a surgical robot giant.

In a previous earnings call in January, J&J Chief Financial Officer Joe Wolk said the company is now looking for opportunities, whether through artificial intelligence or overall infrastructure, to further improve the profitability of medical technology.

The primary goal of the partnership is also to integrate Nvidia's two platforms into Johnson & Johnson's surgical technology portfolio to create better and smarter AI products for its products. At present, Johnson & Johnson Medical Technology's surgical robot products include Monarch natural human cavity treatment robot and VELYS orthopedic surgery robot.

Monarch is a medical robot that passes through the natural cavity of the human body

Monarch is the first approved natural cavity treatment robot in China, which is suitable for treatment through the bronchus.

As the world's leading digital surgery platform, Monarch uses the world's first endoscope technology for full vision, electromagnetic navigation, and virtual reconstruction "three-in-one" navigation. Once the endoscope enters the body, it can continuously display images to provide a comprehensive and clear intraoperative vision. It is also driven by a flexible bronchoscope, allowing easy access to deep lesions. The controller is a remote control shaped like a gamepad that can be operated with precision and flexibility, and the flexible bronchoscope allows 360° Angle adjustment.

VELYS orthopaedic surgical robot

The VELYS orthopedic surgical robot is the first of its kind to have a portable design that can be placed at the surgical bedside, supporting plug and play. Integrated into any operating room environment. The system has good planning ability, optimizes the doctor's surgical process, and ADAPTS to the doctor's surgical habits.

VELYS™ helps surgeons accurately remove bone during total knee replacement, allowing implants to align and position relative to soft tissue without the need for pre-operative imaging. The surgeon does not need to use disposable instruments, greatly reducing the cost of surgery.

Not only a robotic System, but also an integrated smart platform and the ATTUNE Total Knee System, VELYS™ enables a complete solution that assists physicians in assessing the bone anatomy and soft tissue envelope of the knee, assisting in planning the best placement for implant. Balance gap data helps orthopedic surgeons visualize and predict joint stability, and robotic systems assist surgeons in executing surgical plans.

03 Nvidia Cross-border Healthcare

This is not the first time that Nvidia has cooperated with an armed company to jointly layout the medical market.

Previously, three leading medical device startups - Activ Surgical, Moon Surgical and Proximie - have chosen the NVIDIA Clara Holoscan combination running on the IGX platform to power their surgical robotic systems.

Medtronic has also chosen to work with Nvidia to embed its medical-grade Holoscan and IGX platforms into the GI Genius AI-assisted colonoscopy system to provide doctors with AI-enhanced diagnostic images.

Medtronic

In March, Medtronic announced it would work with chipmaker Nvidia to embed its medical-grade real-time AI computing software platform NVIDIA Holoscan and industrial-grade edge AI hardware platform NVIDIA IGX into the GI Genius AI-assisted colonoscopy system. Provide AI-enhanced diagnostic images to doctors. Holoscan helps bring new AI applications to clinical Settings by providing the full-stack infrastructure needed for scalable, software-defined streaming data processing at the edge.

With the NVIDIA Holoscan and IGX platforms, developers can efficiently train and validate AI models within the Cosmo Innovation Center, and then host AI-enabled applications on Medtronic's software-as-a-medical device (SaMD) application marketplace, the GI Genius AI Access platform. It makes it possible for software-defined medical devices.

Activ Surgical

Activ Surgical has selected NVIDIA Clara Holoscan to accelerate the development of its artificial intelligence and augmented reality solutions to enable real-time surgical guidance. The Boston-based company's ActivSight technology allows surgeons to view critical physiological structures and functions that cannot be seen with the naked eye, such as blood flow.

Activ Surgical hopes to revolutionize surgical treatment through augmented reality, artificial intelligence and machine learning technologies. Among them, the company's surgical intelligence platform ActiveEdge can enhance surgeons' surgical decision-making ability through AI-based big data analysis capabilities, thereby reducing unexpected surgical errors; The ActivSight imaging module is designed to seamlessly upgrade existing operating room equipment, act as the "eyes" of the ActiveEdge platform, and allow surgeons to access critical visual data during surgery through AR technology.

Activ Surgical's products have been piloted in several major hospital networks in the United States and Europe, and the ActivSight imaging module will be widely available in 2022.

Moon Surgical

Paris-based robotic surgery company Moon Surgical is designing Maestro, an accessible adaptive surgical assistance robotic system that works with equipment and workflows already in place in the operating room. The company has been able to focus its engineering resources on AI algorithms and other unique features, and adopting Clara Holoscan has saved them time and resources, helping them compress their development schedule.

The Maestro is a collaborative robotic platform (not similar to the Da Vinci surgical robot) that provides the surgeon with stable retraction, camera fixing, and two additional arms, which the surgeon controls at all times and can position in any way needed, safely and reliably. The company is currently developing an intraoperative camera tracking software that will enable the camera arm to automatically track instruments operated by the surgeon. In addition, an embedded room-sensing (equivalent to high-resolution "surveillance") camera tracks the surgeon's movement and operating room environment so that the arm can finally be automatically positioned in the optimal configuration before the procedure begins.

Proximie

London-based Proximie is building a telepresence platform to enable real-time remote surgeon collaboration. Clara Holoscan will allow the company to provide on-premises video processing in the operating room, improving performance for users while maintaining data privacy and reducing cloud computing costs.

Dr. Nadine hach-Haram, founder and CEO of Proximie, said: "We are excited to partner with Nvidia to strengthen the wellness ecosystem and advance our mission of connecting operating rooms globally. "Thanks to this collaboration, we are able to deliver as immersive an experience as possible and provide resilient digital solutions that allow operating room equipment around the world to communicate with each other and gain valuable insights."

Proximie has been deployed in more than 500 operating rooms around the world and tens of thousands of surgical procedures have been recorded to date.

By combining Johnson & Johnson's deep experience in medical technology with NVIDIA's leading technology in artificial intelligence, we can expect to see more innovative medical solutions. The combination of the two giants will not only promote the improvement of the intelligence and data analysis capabilities of medical devices, but also provide unlimited possibilities for the digital transformation and future innovation of the entire medical industry. In this regard, the equipment home will continue to pay attention.

                      END