Mobile Perception Lab

Welcome to the website of the Mobile Perception Lab led by Prof. Laurent Kneip. Our lab was founded in September 2017 and is part of the School of Information Science and Technology (SIST) at ShanghaiTech.

MPL members




Our mission is to develop next-generation 3D perception systems for mobile applications, such as robots, intelligent vehicles, or intelligence augmentation devices (e.g. augmented reality headsets). Through our research, we hope to provide smart mobile machines and devices with the ability to see, to understand their environment using the sense of vision, and to perform useful tasks in the complex, unstructured, and dynamically changing environments in which we live and work. We work on robust, accurate, and efficient localization, 3D scene geometry perception, and scene understanding solutions, where the generated models of the environment cover everything reaching from low-level geometry to higher level aspects such as scene composition, object poses or shapes, dynamics, and semantic meaning. We achieve this goal by combining more classical model and optimization based methods with modern, data-driven techniques. We also make use of novel perception hardware, such as neuro-morphic cameras.


MPL is part of the ShanghaiTech Automation and Robotics Center (STAR Center). Feel free to explore our webpages in order to learn more about our mission.

News

July 29th, 2021

MPL research hot in AR.

After Prof. Kneip was most recently given the honor of a speech at Magic Leap, he is now also giving a speech at Microsoft Research.


July 26th, 2021

Outstanding undergraduate thesis award for Su Hang!

MPL's newly incoming graduate student Su Hang received this year's best undergraduate thesis award here at SIST, ShanghaiTech, for his work on high-speed object tracking with event cameras. He will continue to work on this exciting topic during his graduate studies at MPL.


July 26th, 2021

Welcome Yuan Runze and Su Hang as new graduate students at MPL! Both are from our own undergraduate program.


July 3rd, 2021

Congratulations to MPL member Ouyang Zhanpeng who just graduated as a Master in Computer Science from ShanghaiTech University.

Earlier this year, we already had Wu Peng receiving his degree. In this regard, it is probably also worth mentioning the truly outstanding offers that MPL students may receive on the job market. Peng Wu joined Intel Shanghai, while Ouyang Zhanpeng will join a brand-new AR-oriented division at Beijing KuaiShou Technology Co., Ltd.


July 1st, 2021

And yet another hattrick for MPL, with three accepted papers at IROS'2021.

Please check out our most recent contributions on semantic localization and event-based vision:

  • Monte-Carlo Localization in Underground Parking Lots using Parking Slot Numbers, Li Cui, Chunyan Rong, Jingyi Huang, Andre Rosendo, Laurent Kneip
  • Dynamic Event Camera Calibration, Huang Kun, Yifu Wang, and Laurent Kneip
  • Accurate depth estimation from a hybrid event-RGB stereo setup, Yi-Fan Zuo, Li Cui, Xin Peng, Yanyu Xu, Shenghua Gao, Xia Wang, and Laurent Kneip.

June, 2021

Prof. Kneip is invited to give a talk at WAIC, 2021!


June 1st, 2021

Prof. Kneip appears in the local media

Xinming evening news from Shanghai published a reportage about the work of MPL. Please find the link here


April 6th, 2021

Welcome Wang Yifu, who will join us a post-doc/senior research member here at MPL!

Wang Yifu has been working with Prof. Laurent Kneip since 2016, and already obtained his PhD from the Australian National University under the supervision of Prof. Kneip.


Febraury 28th, 2021

Publication at CVPR 2021!

Ongoing collaboration between Prof. Laurent Kneip and Dr. Ji Zhao from TuSimple leads to new advancements in rotation averaging:

  • Robust and Efficient Global Rotation Averaging with the Burer Monteiro Method, by Y. Chen, J. Zhao, and L. Kneip.

February 28th, 2021

MPL likes hattricks!

Another hattrick publication at ICRA'21. Feel free to check-out our new achievements on vision for self-driving cars and semantic SLAM:

  • Robust SRIF-based LiDAR-IMU Localization for Autonomous Vehicles, by K. Li, Z. Ouyang, L. Hu, D. Hao, and L. Kneip (outperforming LEGO-SLAM!)
  • B-splines for Purely Vision-based Localization and Mapping on Non-holonomic Ground Vehicles, by K. Huang, Y. Wang, and L. Kneip
  • Point Set Registration With Semantic Region Association Using Cascaded Expectation Maximization, by L. Hu, J. Wei, Z. Ouyang, and L. Kneip

January 29th, 2021

New journal publication in Springer Journal of Mathematical Imaging and Vision!

Please check out our latest JMIV publication, where we solve the chicken-egg-problem of scan alignment with extremly low overlap ratios and symmetry plane estimation.


January 15th, 2021

First pure MPL publication in TPAMI!

Glad to announce one of the first fruits of our event-camera based research. Although it is one of the first fruits, it is already a very sweet one: A publication in the IEEE Transactions on Pattern Analysis and Machine Intelligence. Feel free to check out our work on "Globally-Optimal Contrast Maximisation for Event Cameras", by Peng Xin, Ling Gao, Yifu Wang, and Laurent Kneip.


September, 2020

Prof. Kneip is area chair for both ICRA and CVPR 2021!


September 12th, 2020

Big Congratulations to Associate Professor Prof Laurent Kneip!

MPL is immensely pleasured to announce that our director has just been promoted to the tenured Associate Professor level within the School of Information Sciences and Technology here at ShanghaiTech!


August 20th, 2020

Paper presented at ECCV 2020!

MPL is presenting one of their first offsprings on event based computer vision this year at ECCV'2020, a globally optimal solution to event-camera based motion estimation.


July 26th, 2020

Laurent Kneip is an area chair of 3DV 2020 and CVPR 2021!


July 25th, 2020

Laurent Kneip giving a commencement address at the 2020 ShanghaiTech graduation ceremony ... in Chinese!

Please check-out the link here


July 7th, 2020

Excited to announce the first fruits of our work on event based cameras!

MPL just got one paper accepted for ECCV'2020. The paper is called "Globally-Optimal Event Camera Motion Estimation", and it is the first globally optimal solution to continuous motion estimation with an event camera mouned on an AGV.


June 16th, 2020

New publications at ICRA and CVPR!

MPL has had a very successful start into the new year 2020, with a hattrick at ICRA and two publications at CVPR.


May 21st, 2020

Congratulations to Cao Yuchen (Joshua) and Yu Peihong for successfully defending their graduation thesis.

This is a special moment, Cao Yuchen is the first graduate student since the debut of our lab at ShanghaiTech. Please also check out our new alumni section under the people's page, where you will be able to see the faith of past MPL members and collaborators.


August 16, 2019

Laurent Kneip and MPL receive a grant from the International Young Scientist Program from the National Science Foundation of China.

The award apends to another grant received earlier this year from the General Program of the 'Shanghai Innovation in Science and Technology' Natural Science Foundation, and will further support our research on semantic, higher-level SLAM.


July 30, 2019

Paper accepted for 3DV'19

Our innovative application of an aritificial neural network to improve the numerical stability of Groebner basis solvers will be presented orally at 3DV in Quebec City, Canada.


June 24, 2019

New graduate student

Welcome Ling Gao to join MPL, one of ShanghaiTech's own outstanding undergraduate students.


June 20, 2019

Paper accepted for IROS'19

We have an exciting contribution on "articulated" multi-perspective cameras accepted for IROS'19. The case enables motion estimation with non-overlapping cameras distributed over a truck, i.e. tractor and trailor, and handles the fact that these two bodies are moving w.r.t. to each other.


March 28, 2019

Welcoming new members to the group

Welcome Dr Min Li, who is starting as a Research Assistant Professor at MPL. Also welcome Li Cui, who will start as a graduate student at MPL in 2019.


March 12, 2019

Two papers accepted to CVPR'19!

Special congratulations to Huang Kun from MPL on this fine achievement!


February 26, 2019

Congratulations to Dr Zhou Yi!

The first PhD student working under the supervision of Prof Laurent Kneip officially graduated.


January 27, 2019

Laurent Kneip at an NII Shonan meeting

Laurent Kneip was an invited speaker at an NII Shonan meetings on optimization in computer vision, and presented MPL's latest research achievements.


Good start into 2019!

Check out our new contributions to TPAMI and TRO!

TPAMI paper "Minimal case relative pose computation using ray-point-ray features" presents new insights on two-view geometry. TRO paper "Canny-VO: Visual Odometry with RGB-D Cameras based on Geometric 3D-2D Edge Alignment" summarizes our long efforts on semi-dense motion estimation for RGBD cameras.


November 5, 2018

1st prize at the IEEE Australia Postgraduate Student Paper Competition.


September 15, 2018

New graduate student

Welcome Xu Wanting from Xinjiang University to join MPL.


July 9, 2018

ECCV paper on event-based vision accepted!

The fruits of our collaboration with RPG from University of Zurich.


June 25, 2018

New PhD student

Peng Xin from CAS has joined MPL as a new PhD student. Welcome to the team!


June 14, 2018

PAMI paper accepted

Our paper on globally optimal correspondence-less camera resectioning winning a Marr Prize Honourable Mention has been accepted to PAMI!.


March 30, 2018

Oral at CVPR'18!

Our work on globally optimal, non-minimal calibrated relative pose will be presented orally at CVPR'18.


March 23, 2018

New graduate student

Welcome Zhanpeng Ouyang from Nanchang University to join our team!


March 23, 2018

MPL web presence goes live!

We are excited to get started, and here is where you will be able to check out on our progress as we go. We'd love to get feedback from you!


December, 2017

Marr Prize Honourable Mention

Our ICCV'17 work "Globally-Optimal Inlier Set Maximisation for Simultaneous Camera Pose and Feature Correspondence" received a Marr Prize Honourable Mention! The Marr Prize is the most prestigious best paper award in the computer vision community.


Video Highlights

August 21, 2020

Video of our ECCV'2020 contribution

The video of our ECCV'2020 paper on Globally Optimal Event Camera Motion Estimation is now online. The method provides a globally optimal solution to contrast maximization using branch and bound. Links: Youtube, Bilibili, and PDF.