Elf: Accelerate High-resolution Mobile Deep Vision with Content-aware Parallel Offloading

Invention Summary:

  As mobile devices continuously generate streams of images and videos, a new class of mobile deep vision applications is rapidly emerging, which usually involve the use of deep neural networks on the multimedia data in real-time. To support such applications, the ability to offload the computation, especially the inference processing, to edge clouds has proved effective. Existing solutions often assume a dedicated and powerful server exists, to which the entire inference can be offloaded. In reality, it may not be possible to find such a server and therefore a need to make do with less powerful ones.

    To address these more practical situations, Rutgers researchers propose a solution that partitions the video frame and offloads the partial inference tasks to multiple servers for parallel processing. A framework, entitled “Elf”  accelerates the mobile deep vision applications with server provisioning through the parallel offloading.

    Elf has been implemented and evaluated upon Linux and Android platforms using four commercial mobile devices and three deep vision applications with ten state-of-the-art models. The comprehensive experiments show that Elf can speed up the applications by 4.85× with saving bandwidth usage by 52.6%, while with <1% application accuracy sacrifice.  

Market Applications:

Applications that operate on mobile devices and benefit from Edge Computing for fast performance:

• Autonomous Vehicles/Mobile robots
• Video Surveillance
• Augmented Reality
• Virtual Reality
• Smart Cities, etc.

Advantages:

• Able to minimize the end-to-end latency of mobile deep vision applications
• Able to ensure the accuracy at the same time

Intellectual Property & Development Status: Patent pending. Available for licensing and/or research collaboration.