Document Open Access Logo

Introduction to the Special Issue on Embedded Systems for Computer Vision

Authors Samarjit Chakraborty , Qing Rao

PDF
Thumbnail PDF

File

LITES.8.1.0.pdf
  • Filesize: 478 kB
  • 8 pages

Document Identifiers

Author Details

Samarjit Chakraborty
  • The University of North Carolina (UNC) at Chapel Hill, US
Qing Rao
  • Momenta Europe GmbH, Stuttgart, Germany

Cite AsGet BibTex

LITES, Volume 8, Issue 1: Special Issue on Embedded Systems for Computer Vision, pp. 0:i-0:viii, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022)
https://doi.org/10.4230/LITES.8.1.0

Abstract

We provide a broad overview of some of the current research directions at the intersection of embedded systems and computer vision, in addition to introducing the papers appearing in this special issue. Work at this intersection is steadily growing in importance, especially in the context of autonomous and cyber-physical systems design. Vision-based perception is almost a mandatory component in any autonomous system, but also adds myriad challenges like, how to efficiently implement vision processing algorithms on resource-constrained embedded architectures, and how to verify the functional and timing correctness of these algorithms. Computer vision is also crucial in implementing various smart functionality like security, e.g., using facial recognition, or monitoring events or traffic patterns. Some of these applications are reviewed in this introductory article. The remaining articles featured in this special issue dive into more depth on a few of them.

Subject Classification

ACM Subject Classification
  • Computer systems organization → Embedded and cyber-physical systems
Keywords
  • Embedded systems
  • Computer vision
  • Cyber-physical systems
  • Computer architecture

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Tanya Amert and James H. Anderson. Cupid^rt: Detecting improper GPU usage in real-time applications. In 24th IEEE International Symposium on Real-Time Distributed Computing (ISORC), 2021. Google Scholar
  2. Tanya Amert, Michael Balszun, Martin Geier, F. Donelson Smith, James H. Anderson, and Samarjit Chakraborty. Timing-predictable vision processing for autonomous systems. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 2021. Google Scholar
  3. Tanya Amert, Zelin Tong, Sergey Voronov, Joshua Bakita, F. Donelson Smith, and James H. Anderson. Timewall: Enabling time partitioning for real-time multicore+accelerator platforms. In 42nd IEEE Real-Time Systems Symposium (RTSS), 2021. Google Scholar
  4. Michael Balszun, Martin Geier, and Samarjit Chakraborty. Predictable vision for autonomous systems. In 23rd IEEE International Symposium on Real-Time Distributed Computing (ISORC), 2020. Google Scholar
  5. Wanli Chang and Samarjit Chakraborty. Resource-aware automotive control systems design: A cyber-physical systems approach. Found. Trends Electron. Des. Autom., 10(4):249-369, 2016. Google Scholar
  6. Wanli Chang, Debayan Roy, Shuai Zhao, Anuradha Annaswamy, and Samarjit Chakraborty. Cps-oriented modeling and control of traffic signals using adaptive back pressure. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 2020. Google Scholar
  7. Sayandip De, Sajid Mohamed, Konstantinos Bimpisidis, Dip Goswami, Twan Basten, and Henk Corporaal. Approximation trade offs in an image-based control system. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 2020. Google Scholar
  8. Sayandip De, Sajid Mohamed, Dip Goswami, and Henk Corporaal. Approximation-aware design of an image-based control system. IEEE Access, 8:174568-174586, 2020. Google Scholar
  9. Glenn A. Elliott, Bryan C. Ward, and James H. Anderson. GPUSync: A framework for real-time GPU management. In 34th IEEE Real-Time Systems Symposium (RTSS), 2013. Google Scholar
  10. Glenn A. Elliott, Kecheng Yang, and James H. Anderson. Supporting real-time computer vision workloads using OpenVX on Multicore+GPU platforms. In IEEE Real-Time Systems Symposium (RTSS), 2015. Google Scholar
  11. Martin Geier, Marian Brändle, and Samarjit Chakraborty. Insert & save: Energy optimization in IP core integration for FPGA-based real-time systems. In 27th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS), 2021. Google Scholar
  12. Martin Geier, Marian Brändle, Dominik Faller, and Samarjit Chakraborty. Debugging FPGA-accelerated real-time systems. In IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS), 2020. Google Scholar
  13. Martin Geier, Dominik Faller, Marian Brändle, and Samarjit Chakraborty. Cost-effective energy monitoring of a Zynq-based real-time system including dual Gigabit Ethernet. In 27th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), 2019. Google Scholar
  14. Martin Geier, Florian Pitzl, and Samarjit Chakraborty. GigE vision data acquisition for visual servoing using SG/DMA proxying. In 14th ACM/IEEE Symposium on Embedded Systems for Real-Time Multimedia (ESTIMedia), 2016. Google Scholar
  15. Abhinav Goel, Caleb Tung, Xiao Hu, George K. Thiruvathukal, James C. Davis, and Yung-Hsiang Lu. Efficient computer vision on edge devices with pipeline-parallel hierarchical neural networks. In 27th Asia and South Pacific Design Automation Conference (ASP-DAC), 2022. Google Scholar
  16. Kees Goossens, Arnaldo Azevedo, Karthik Chandrasekar, Manil Dev Gomony, Sven Goossens, Martijn Koedam, Yonghui Li, Davit Mirzoyan, Anca Mariana Molnos, Ashkan Beyranvand Nejad, Andrew Nelson, and Shubhendu Sinha. Virtual execution platforms for mixed-time-criticality systems: the compsoc architecture and design flow. SIGBED Rev., 10(3):23-34, 2013. Google Scholar
  17. Dip Goswami, Reinhard Schneider, and Samarjit Chakraborty. Re-engineering cyber-physical control applications for hybrid communication protocols. In Design, Automation and Test in Europe (DATE), 2011. Google Scholar
  18. Dip Goswami, Reinhard Schneider, and Samarjit Chakraborty. Relaxing signal delay constraints in distributed embedded controllers. IEEE Trans. Control. Syst. Technol., 22(6):2337-2345, 2014. Google Scholar
  19. Clara Hobbs, Debayan Roy, Parasara Sridhar Duggirala, F. Donelson Smith, Soheil Samii, James H. Anderson, and Samarjit Chakraborty. Perception computing-aware controller synthesis for autonomous systems. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 2021. Google Scholar
  20. Omar W Ibraheem, Arif Irwansyah, Jens Hagemeyer, Mario Porrmann, and Ulrich Rueckert. A resource-efficient multi-camera GiGE vision IP core for embedded vision processing platforms. In IEEE International Conference on ReConFigurable Computing and FPGAs (ReConFig), 2015. Google Scholar
  21. Chaitanya Jugade, Daniel Hartgers, Phan Dúc Anh, Sajid Mohamed, Mojtaba Haghi, Dip Goswami, Andrew Nelson, Gijs van der Veen, and Kees Goossens. An evaluation framework for vision-in-the-loop motion control systems. In 27th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 2022. Google Scholar
  22. Anis Koubaa, Adel Ammar, Anas Kanhouch, and Yasser AlHabashi. Cloud versus edge deployment strategies of real-time face recognition inference. IEEE Transactions on Network Science and Engineering, 9(1):143-160, 2021. Google Scholar
  23. Friedrich Kruber, Eduardo Sánchez Morales, Samarjit Chakraborty, and Michael Botsch. Vehicle position estimation with aerial imagery from unmanned aerial vehicles. In IEEE Intelligent Vehicles Symposium (IV), 2020. Google Scholar
  24. Friedrich Kruber, Jonas Wurst, Eduardo Sánchez Morales, Samarjit Chakraborty, and Michael Botsch. Unsupervised and supervised learning with the random forest algorithm for traffic scenario clustering and classification. In IEEE Intelligent Vehicles Symposium (IV), 2019. Google Scholar
  25. Zhen Liu, Wenjie Lin, Xinpeng Li, Qing Rao, Ting Jiang, Mingyan Han, Haoqiang Fan, Jian Sun, and Shuaicheng Liu. Adnet: Attention-guided deformable convolutional network for high dynamic range imaging. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2021. Google Scholar
  26. Dipan Kumar Mandal, Jagadeesh Sankaran, Akshay Gupta, Kyle Castille, Shraddha Gondkar, Sanmati Kamath, Pooja Sundar, and Alan Phipps. An embedded vision engine (EVE) for automotive vision processing. In IEEE International Symposium on Circuits and Systems (ISCAS), 2014. Google Scholar
  27. Alejandro Masrur, Sebastian Drössler, Thomas Pfeuffer, and Samarjit Chakraborty. VM-based real-time services for automotive control applications. In 16th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), 2010. Google Scholar
  28. Mahmoud Méribout, Asma Baobaid, Mohamed Ould-Khaoua, Varun Kumar Tiwari, and Juan Pablo Pena. State of art iot and edge embedded systems for real-time machine vision applications. IEEE Access, 10:58287-58301, 2022. Google Scholar
  29. Seifeddine Messaoud, Soulef Bouaafia, Amna Maraoui, Ahmed Chiheb Ammari, Lazhar Khriji, and Mohsen Machhout. Deep convolutional neural networks-based hardware-software on-chip system for computer vision application. Comput. Electr. Eng., 98:107671, 2022. Google Scholar
  30. Sajid Mohamed, Dip Goswami, Vishak Nathan, Raghu Rajappa, and Twan Basten. A scenario- and platform-aware design flow for image-based control systems. Microprocess. Microsystems, 75:103037, 2020. Google Scholar
  31. Sajid Mohamed, Nilay Saraf, Daniele Bernardini, Dip Goswami, Twan Basten, and Alberto Bemporad. Adaptive predictive control for pipelined multiprocessor image-based control systems considering workload variations. In 59th IEEE Conference on Decision and Control (CDC), 2020. Google Scholar
  32. Burhan Ahmad Mudassar, Priyabrata Saha, Yun Long, Mohammad Faisal Amir, Evan Gebhardt, Taesik Na, Jong Hwan Ko, Marilyn Wolf, and Saibal Mukhopadhyay. CAMEL: an adaptive camera with embedded machine learning-based sensor parameter control. IEEE J. Emerg. Sel. Topics Circuits Syst., 9(3):498-508, 2019. Google Scholar
  33. The Khronos Group, OpenVX: Portable, power efficient vision processing. URL: https://www.khronos.org/openvx/.
  34. Nathan Otterness and James H. Anderson. AMD GPUs as an alternative to NVIDIA for supporting real-time workloads. In 32nd Euromicro Conference on Real-Time Systems (ECRTS), volume 165 of LIPIcs, pages 10:1-10:23. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020. Google Scholar
  35. Qing Rao and Samarjit Chakraborty. Efficient lossless compression for depth information in traffic scenarios. Multim. Syst., 25(4):293-306, 2019. Google Scholar
  36. Qing Rao and Samarjit Chakraborty. In-vehicle object-level 3D reconstruction of traffic scenes. IEEE Trans. Intell. Transp. Syst., 22(12):7747-7759, 2021. Google Scholar
  37. Qing Rao, Christian Grünler, Markus Hammori, and Samarjit Chakraborty. Design methods for augmented reality in-vehicle infotainment systems. In 51st Annual Design Automation Conference (DAC), 2014. Google Scholar
  38. Qing Rao, Christian Grünler, Markus Hammori, and Samarjit Chakraborty. Stixel on the bus: An efficient lossless compression scheme for depth information in traffic scenarios. In 20th Anniversary International Conference of MultiMedia Modeling (MMM), volume 8325 of Lecture Notes in Computer Science. Springer, 2014. Google Scholar
  39. Qing Rao, Lars Krüger, and Klaus Dietmayer. Monocular 3D shape reconstruction using deep neural networks. In IEEE Intelligent Vehicles Symposium (IV), 2016. Google Scholar
  40. Qing Rao, Tobias Tropper, Christian Grünler, Markus Hammori, and Samarjit Chakraborty. AR-IVI - implementation of in-vehicle augmented reality. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2014. Google Scholar
  41. Debayan Roy, Clara Hobbs, James H. Anderson, Marco Caccamo, and Samarjit Chakraborty. Timing debugging for cyber-physical systems. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 2021. Google Scholar
  42. Debayan Roy, Licong Zhang, Wanli Chang, Sanjoy K. Mitter, and Samarjit Chakraborty. Semantics-preserving cosynthesis of cyber-physical systems. Proc. IEEE, 106(1):171-200, 2018. Google Scholar
  43. Kruttidipta Samal, Hemant Kumawat, Priyabrata Saha, Marilyn Wolf, and Saibal Mukhopadhyay. Task-driven rgb-lidar fusion for object tracking in resource-efficient autonomous system. IEEE Trans. Intell. Veh., 7(1):102-112, 2022. Google Scholar
  44. Kruttidipta Samal, Marilyn Wolf, and Saibal Mukhopadhyay. Closed-loop approach to perception in autonomous system. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 2021. Google Scholar
  45. Róbinson Medina Sánchez, Juan Valencia, Sander Stuijk, Dip Goswami, and Twan Basten. Designing a controller with image-based pipelined sensing and additive uncertainties. ACM Trans. Cyber Phys. Syst., 3(3):33:1-33:26, 2019. Google Scholar
  46. Sebastian Schmidt, Qing Rao, Julian Tatsch, and Alois C. Knoll. Advanced active learning strategies for object detection. In IEEE Intelligent Vehicles Symposium (IV), 2020. Google Scholar
  47. Reinhard Schneider, Dip Goswami, Alejandro Masrur, Martin Becker, and Samarjit Chakraborty. Multi-layered scheduling of mixed-criticality cyber-physical systems. J. Syst. Archit., 59(10-D):1215-1230, 2013. Google Scholar
  48. Shreejith Shanker, Philipp Mundhenk, Andreas Ettner, Suhaib A. Fahmy, Sebastian Steinhorst, Martin Lukasiewycz, and Samarjit Chakraborty. VEGa: A high performance vehicular ethernet gateway on hybrid FPGA. IEEE Trans. Computers, 66(10):1790-1803, 2017. Google Scholar
  49. Ponnan Suresh, Saravanakumar Umathurai, Celestine Iwendi, Senthilkumar Mohan, and Gautam Srivastava. Field-programmable gate arrays in a low power vision system. Comput. Electr. Eng., 90:106996, 2021. Google Scholar
  50. George K. Thiruvathukal and Yung-Hsiang Lu. Efficient computer vision for embedded systems. Computer, 55(4):15-19, 2022. Google Scholar
  51. Caleb Tung, Abhinav Goel, Fischer Bordwell, Nick Eliopoulos, Xiao Hu, Yung-Hsiang Lu, and George K. Thiruvathukal. Why accuracy is not enough: The need for consistency in object detection. IEEE Multim., 29(3):8-16, 2022. Google Scholar
  52. R. Udendhran, M. Balamurugan, Annamalai Suresh, and R. Varatharajan. Enhancing image processing architecture using deep learning for embedded vision systems. Microprocess. Microsystems, 76, 2020. Google Scholar
  53. Zhilu Wang, Chao Huang, Yixuan Wang, Clara Hobbs, Samarjit Chakraborty, and Qi Zhu. Bounding perception neural network uncertainty for safe control of autonomous systems. In Design, Automation & Test in Europe Conference & Exhibition (DATE), 2021. Google Scholar
  54. Peter Waszecki, Philipp Mundhenk, Sebastian Steinhorst, Martin Lukasiewycz, Ramesh Karri, and Samarjit Chakraborty. Automotive electrical and electronic architecture security via distributed in-vehicle traffic monitoring. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., 36(11):1790-1803, 2017. Google Scholar
  55. Ming Yang, Tanya Amert, Kecheng Yang, Nathan Otterness, James H. Anderson, F. Donelson Smith, and Shige Wang. Making openvx really "real time". In IEEE Real-Time Systems Symposium (RTSS), 2018. Google Scholar
  56. Ming Yang, Shige Wang, Joshua Bakita, Thanh Vu, F. Donelson Smith, James H. Anderson, and Jan-Michael Frahm. Re-thinking CNN frameworks for time-sensitive autonomous-driving applications: Addressing an industrial challenge. In 25th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS), 2019. Google Scholar
  57. Anand Yeolekar, Ravindra Metta, Clara Hobbs, and Samarjit Chakraborty. Checking scheduling-induced violations of control safety properties. In 20th International Symposium on Automated Technology for Verification and Analysis (ATVA), volume 13505 of Lecture Notes in Computer Science. Springer, 2022. Google Scholar
  58. Junxing Zhang, Shuo Yang, Chunjuan Bo, and Zhiyuan Zhang. Vehicle logo detection based on deep convolutional networks. Comput. Electr. Eng., 90:107004, 2021. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact