Vol. 8 No. 1 (2022): Special Issue on Embedded Systems for Computer Vision
Special Issue on Embedded Systems for Computer Vision

Micro- and Macroscopic Road Traffic Analysis using Drone Image Data

Friedrich Kruber
Technische Hochschule Ingolstadt, Esplanade 10, Ingolstadt, Germany
Eduardo Sánchez Morales
Technische Hochschule Ingolstadt, Esplanade 10, Ingolstadt, Germany
Robin Egolf
Technische Hochschule Ingolstadt, Esplanade 10, Ingolstadt, Germany
Jonas Wurst
Technische Hochschule Ingolstadt, Esplanade 10, Ingolstadt, Germany
Samarjit Chakraborty
University of North Carolina at Chapel Hill (UNC), Department of Computer Science, NC 27599, USA
Michael Botsch
Technische Hochschule Ingolstadt, Esplanade 10, Ingolstadt, Germany

Published 2022-11-16

Keywords

  • traffic data analysis,
  • trajectory data,
  • drone image data

How to Cite

[1]
Kruber, F., Sánchez Morales, E., Egolf, R., Wurst, J., Chakraborty, S. and Botsch, M. 2022. Micro- and Macroscopic Road Traffic Analysis using Drone Image Data. Leibniz Transactions on Embedded Systems. 8, 1 (Nov. 2022), 02:1–02:27. DOI:https://doi.org/10.4230/LITES.8.1.2.

Abstract

The current development in the drone technology, alongside with machine learning based image processing, open new possibilities for various applications. Thus, the market volume is expected to grow rapidly over the next years. The goal of this paper is to demonstrate the capabilities and limitations of drone based image data processing for the purpose of road traffic analysis.
In the first part a method for generating microscopic traffic data is proposed. More precisely, the state of vehicles and the resulting trajectories are estimated. The method is validated by conducting experiments with reference sensors and proofs to achieve precise vehicle state estimation results. It is also shown, how the computational effort can be reduced by incorporating the tracking information into a neural network. A discussion on current limitations supplements the findings. By collecting a large number of vehicle trajectories, macroscopic statistics, such as traffic flow and density can be obtained from the data. In the second part, a publicly available drone based data set is analyzed to evaluate the suitability for macroscopic traffic modeling. The results show that the method is well suited for gaining detailed information about macroscopic statistics, such as traffic flow dependent time headway or lane change occurrences. In conclusion, this paper presents methods to exploit the remarkable opportunities of drone based image processing for joint macro- and microscopic traffic analysis.

References

  1. Daten und Fakten: Autobahn-Unfälle, 2010.
  2. Runter vom Gas, Unfallursachen, 2018.
  3. Mujahid Abdulrahim. On the dynamics of automobile drifting. SAE Mobilus, April 2006. URL: https://doi.org/10.4271/2006-01-1019.
  4. Seyed Majid Azimi, Eleonora Vig, Reza Bahmanyar, Marco Körner, and Peter Reinartz. Towards multi-class object detection in unconstrained remote sensing imagery, 2018. URL: http://arxiv.org/abs/1807.02700.
  5. S. Bae and K. Yoon. Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 1218-1225, 2014. URL: https://doi.org/10.1109/CVPR.2014.159.
  6. Emmanouil Barmpounakis and Nikolas Geroliminis. On the new era of urban traffic monitoring with massive drone data: The pneuma large-scale field experiment. Transportation Research Part C: Emerging Technologies, 111:50-71, 2020. URL: https://doi.org/10.1016/j.trc.2019.11.023.
  7. Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. SURF: Speeded Up Robust Features. In Computer Vision - ECCV, 2019.
  8. A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft. Simple online and realtime tracking. In 2016 IEEE International Conference on Image Processing (ICIP), 2016.
  9. Peter J. Bickel, Chao Chen, Jaimyoung Kwon, John Rice, Erik van Zwet, and Pravin Varaiya. Measuring Traffic. Statistical Science, 22(4), 2007.
  10. Laura Bieker-Walz. Wie kann eine Verkehrssimulation den Rettungsdienst unterstützen? In WAW DLR.Open II, oktober 2017. URL: https://elib.dlr.de/114841/.
  11. Ilker Bozcan and Erdal Kayaan. Au-air: A multi-modal unmanned aerial vehicle dataset for low altitude traffic surveillance. In IEEE International Conference on Robotics and Automation (ICRA), 2020.
  12. Antonia Breuer, Jan-Aike Termöhlen, Silviu Homoceanu, and Tim Fingscheidt. Opendd: A large-scale roundabout drone dataset. In Proceedings of International Conference on Intelligent Transportation Systems, September 2020.
  13. Bundesverband der Deutschen Luftverkehrswirtschaft. Analyse des deutschen Drohnenmarktes. URL: https://www.bdl.aero/de/publikation/analyse-des-deutschen-drohnenmarktes/.
  14. Fritz Busch. Spurbelastungen und Häufigkeit von Spurwechseln auf einer dreispurigen BAB-Richtungsfahrbahn. ATZ - Automobiltechnische Zeitschrift, June 1984. URL: https://doi.org/10.1007/s35148-012-0485-x.
  15. CATT (University of Maryland. Traffic Flow Measures Implementation Guide, 2008. URL: http://www.catt.umd.edu/sites/default/files/documents/traffic_flow_measure_guidelines_v8.pdf.
  16. Bo-Chiuan Chen and Feng-Chi Hsieh. Sideslip angle estimation using extended kalman filter. Vehicle System Dynamics - VEH SYST DYN, 46, September 2008. URL: https://doi.org/10.1080/00423110801958550.
  17. C. Feichtenhofer, A. Pinz, and A. Zisserman. Detect to track and track to detect. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 3057-3065, 2017. URL: https://doi.org/10.1109/ICCV.2017.330.
  18. B Grienshields. The photographic method of studying traffic behavior. In Proceedings of the Thirteenth Annual Meeting of the Highway Research Board, 1933.
  19. Giuseppe Guido, Vincenzo Gallelli, Daniele Rogano, and Alessandro Vitale. Evaluating the accuracy of vehicle tracking data obtained from Unmanned Aerial Vehicles. International Journal of Transportation Science and Technology, 2016.
  20. Hall, Fred. TRAFFIC STREAM CHARACTERISTICS, 1996.
  21. Jörg Haus and Norbert Lauinger. Optische gitter: Die abbildung der realität – 75 jahre berührungslose dynamische meßtechnik auf der basis optischer gitter. Laser Technik Journal, 4:43-47, April 2007. URL: https://doi.org/10.1002/latj.200790155.
  22. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask R-CNN. In Proceedings of the International Conference on Computer Vision (ICCV), 2017.
  23. Dirk Helbing. Traffic and related self-driven many-particle systems. Rev. Mod. Phys., 73:1067-1141, December 2001. URL: https://doi.org/10.1103/RevModPhys.73.1067.
  24. R.E. Kalman. A new approach to linear filtering and prediction problems, 1960.
  25. Kai Kang, Hongsheng Li, Tong Xiao, Wanli Ouyang, Junjie Yan, Xihui Liu, and Xiaogang Wang. Object detection in videos with tubelet proposal networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017. URL: https://doi.org/10.1109/cvpr.2017.101.
  26. Kai Kang, Wanli Ouyang, Hongsheng Li, and Xiaogang Wang. Object detection from video tubelets with convolutional neural networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. URL: https://doi.org/10.1109/cvpr.2016.95.
  27. Ebrahim Karami, Siva Prasad, and Mohamed Shehata. Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images, 2017.
  28. Robert Krajewski, Julian Bock, Laurent Kloeker, and Lutz Eckstein. The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems. In IEEE 21st International Conference on Intelligent Transportation Systems (ITSC), 2018.
  29. Stefan Krauß. Microscopic Modeling of Traffic Flow:Investigation of Collision Free Vehicle Dynamics.
  30. F. Kruber, E. Sánchez Morales, S. Chakraborty, and M. Botsch. Vehicle Position Estimation with Aerial Imagery from Unmanned Aerial Vehicles. In IEEE Intelligent Vehicles Symposium (IV), 2020.
  31. H. W. Kuhn and Bryn Yaw. The hungarian method for the assignment problem. Naval Res. Logist. Quart, pages 83-97, 1955.
  32. C. Kyrkou, G. Plastiras, T. Theocharides, S. I. Venieris, and C. Bouganis. DroNet: Efficient convolutional neural network detector for real-time UAV applications. In Design, Automation Test in Europe Conference Exhibition (DATE), 2018.
  33. Lambda Labs. Deep Learning GPU Benchmarks, 2021. URL: https://lambdalabs.com/gpu-benchmarks.
  34. Qingpeng Li, Lichao Mou, Qizhi Xu, Yun Zhang, and Xiao Xiang Zhu. R^3-Net: A Deep Network for Multi-oriented Vehicle Detection in Aerial Images and Videos. CoRR, 2018. URL: http://arxiv.org/abs/1808.05560.
  35. H. Lietz, Petzoldt T., M. Henning, J. Haupt, G. Wanielik, J. Krems, H. Mosebach, J. Schomerus, M. Baumann, and U. Noyer. Methodische und technische Aspekte einer Naturalistic Driving Study. FAT Schriftenreihe, 2011.
  36. Thomas Lillesand, Ralph W. Kiefer, and Jonathan Chipman. Remote Sensing and Image Interpretation, volume 5. Wiley, 2003.
  37. T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature Pyramid Networks for Object Detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  38. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common Objects in Context. In Computer Vision - ECCV, 2014.
  39. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
  40. A.L. Majdik, C. Till, and D. Scaramuzza. The zurich urban micro aerial vehicle dataset. International Journal of Robotics Research, 2017.
  41. L. Mou and X. X. Zhu. Vehicle Instance Segmentation From Aerial Image and Video Using a Multitask Learning Residual Fully Convolutional Network. IEEE Transactions on Geoscience and Remote Sensing, 2018.
  42. Matthias Mueller, Neil Smith, and Bernard Ghanem. A benchmark and simulator for uav tracking. In Computer Vision - ECCV 2016. Springer International Publishing, 2016.
  43. Lutz Neubert. Statistische Analyse von Verkehrsdaten und die Modellierung von Verkehrsfluss mittels zellularer Automaten. PhD thesis, Universität Duisburg, 2000.
  44. Pix4D SA. Example projects - real photogrammetry data (https://www.pix4d.com). URL: https://www.pix4d.com.
  45. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You Only Look Once: Unified, Real-Time Object Detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  46. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Advances in Neural Information Processing Systems 28. Curran Associates, Inc., 2015.
  47. A. Robicquet, A. Sadeghian, A. Alahi, and S. Savarese. Human Trajectory Prediction In Crowded Scenes. In European Conference on Computer Vision (ECCV), 2016.
  48. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. CoRR, 2015.
  49. E. Rublee, V. Rabaud, K. Konolige, and G. Bradski. Orb: An efficient alternative to sift or surf. In 2011 International Conference on Computer Vision, 2011.
  50. E. Sánchez Morales, F. Kruber, M. Botsch, B. Huber, and A. García Higuera. Accuracy Characterization of the Vehicle State Estimation from Aerial Imagery. In IEEE Intelligent Vehicles Symposium (IV), 2020.
  51. Patrick Schneider, Martin Butz, Christian Heinzemann, Jens Oehlerking, and Matthias Woehrle. Scenario-based threat metric evaluation based on the highd dataset. In IEEE Intelligent Vehicles Symposium (IV), 2020.
  52. Dieter Schramm, Manfred Hiller, Roberto Bardini, et al. Modellbildung und Simulation der Dynamik von Kraftfahrzeugen, volume 124. Springer, 2010.
  53. Marc René Zofka Tobias Fleck, Sven Ochs and J. Marius Zöllner. (accepted) robust tracking of reference trajectories for autonomous driving in intelligent roadside infrastructure. In Intelligent Vehicles Symposium (IV) 2020, oktober 2020.
  54. Philip Torr and A. Zisserman. MLESAC: A New Robust Estimator with Application to Estimating Image Geometry. Computer Vision and Image Understanding, June 2000.
  55. Verband der TÜV e.V. Merkblatt 751, 2008.
  56. Bas Vergouw, Huub Nagel, Geert Bondt, and Bart Custers. Drone Technology: Types, Payloads, Applications, Frequency Spectrum Issues and Future Developments, pages 21-45. T.M.C. Asser Press, 2016. URL: https://doi.org/10.1007/978-94-6265-132-6_2.
  57. L. Wang, W. Ouyang, X. Wang, and H. Lu. Visual tracking with fully convolutional networks. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 3119-3127, 2015. URL: https://doi.org/10.1109/ICCV.2015.357.
  58. Wei Zhan, Liting Sun, Di Wang, Haojie Shi, Aubrey Clausse, Maximilian Naumann, Julius Kümmerle, Hendrik Königshof, Christoph Stiller, Arnaud de La Fortelle, and Masayoshi Tomizuka. INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps. arXiv, 2019. URL: http://arxiv.org/abs/1910.03088.
  59. Pengfei Zhu, Longyin Wen, Xiao Bian, Ling Haibin, and Qinghua Hu. Vision Meets Drones: A Challenge. arXiv preprint, 2018. URL: http://arxiv.org/abs/1804.07437.