Abstract

Real-time systems are widely used from the automotive industry to the aerospace industry. The scientists, researchers, and engineers who develop real-time platforms, worst-case execution time analysis methods and tools need to compare their solutions to alternatives. For this purpose, they use benchmark applications. Today many of our computing systems are multicore and/or multiprocessor systems. Therefore, to be able to compare the effectiveness of real-time platforms, worst-case execution time analysis methods and tools, the research community need multi-threaded benchmark applications which scale on multicore and/or multiprocessor systems. In this paper, we present the first version of PBench, a parallel, real-time benchmark suite. PBench includes different types of multi-threaded applications which implement various algorithms from searching to sorting, matrix multiplication to probability distribution calculation. In addition, PBench provides single-threaded versions of all programs to allow side by side comparisons.

References

[1] S. E. Sim, S. Easterbrook, and R. C. Holt, “Using benchmarking to advance research: a challenge to software engineering,” 25th Int. Conf. Softw. Eng. 2003. Proceedings., pp.74–83, 2003.
[2] A. Grama, A. Gupta, G. Karypis, and V. Kumar, “Introduction to Parallel Computing, Second Edition,” Addison-Wesley, 2003, p. 656.
[3] B. K. David and H. Wen-mei W., Programming Massively Parallel Processors, 2nd Edition. Morgan Kaufmann, 2012.
[4] J. Diaz, C. Muñoz-Caro, and A. Niño, “A survey of parallel programming models and tools in the multi and many-core era,” IEEE Trans. Parallel Distrib. Syst., vol. 23, no. 8, pp. 1369–1386, 2012.
[5] H. Kopetz, Real-Time Systems. Boston, MA: Springer US, 2011.
[6] R. Wilhelm et al., “The worst-case execution-time problem—overview of methods and survey of tools,” ACM Trans. Embed. Comput. Syst., vol. 7, no. 3, pp. 1–53, Apr. 2008.
[7] J. Abella et al., “WCET analysis methods: Pitfalls and challenges on their trustworthiness,” in 10th IEEE International Symposium on Industrial Embedded Systems (SIES), 2015, pp. 1–10.
[8] J. Abella, D. Hardy, I. Puaut, E. Quinones, and F. J. Cazorla, “On the comparison of deterministic and probabilistic WCET estimation techniques,” in Proceedings - Euromicro Conference on Real-Time Systems, 2014, pp. 266–275.
[9] S. Altmeyer, L. Cucu-Grosjean, and R. I. Davis, “Static probabilistic timing analysis for real-time systems using random replacement caches,” Real-Time Syst., vol. 51, no. 1, pp.77–123, 2015.
[10] L. Kosmidis et al., “Fitting processor architectures for measurement-based probabilistic timing analysis,” Microprocess. Microsyst., vol. 47, pp. 287–302, 2016.
[11] F. J. Cazorla et al., “PROXIMA: Improving Measurement-Based Timing Analysis through Randomisation and Probabilistic Analysis,” in 2016 Euromicro Conference on Digital System Design (DSD), 2016, pp. 276–285.
[12] S. Altmeyer and R. I. Davis, “On the correctness, optimality and precision of Static Probabilistic Timing Analysis,” in Design, Automation & Test in Europe Conference & Exhibition (DATE), 2014, 2014, pp. 1–6.
[13] “Rapita Systems homepage.” [Online]. Available: https://www.rapitasystems.com. [Accessed: 13-Sep-2018].
[14] “AbsInt homepage.” [Online]. Available: https://www.absint.com. [Accessed: 13-Sep2018].
[15] J. Gustafsson, A. Betts, A. Ermedahl, and B. Lisper, “The mälardalen WCET benchmarks: Past, present and future,” in 10th International Workshop on Worst-Case Execution Time Analysis, WCET 2010, 2010, vol. 15, pp. 136–146.
[16] “The Mälardalen WCET Benchmarks homepage.” [Online]. Available: http://www.mrtc.mdh.se/projects/wcet/benchmarks.html. [Accessed: 13-Aug-2018].
[17] H. Falk et al., “TACLeBench: A Benchmark Collection to Support Worst-Case Execution Time Research,” 16th Int. Work. Worst-Case Exec. Time Anal. (WCET 2016), vol. i, no. 2, p. 10, 2016.
[18] F. Nemer, H. Cassé, P. Sainrat, J.-P. Bahsoun, and M. De Michiel, “PapaBench: a Free Real-Time Benchmark,” in 6th International Workshop on Worst-Case Execution Time Analysis (WCET’06), 2006, vol. 4.
[19] “Sakarya University, Faculty of Computer and Information Sciences, Department of Software Engineering, Real-Time Systems Research Laboratory homepage.” [Online]. Available: http://www.rtsrlab.sakarya.edu.tr. [Accessed: 13-Aug-2018].
[20] “Spec - Standard Performance Evaluation Corporation homepage.” [Online]. Available: https://www.spec.org. [Accessed: 13-Aug-2018].
[21] X. Tian et al., “BigDataBench-S: An open-source scientific big data benchmark suite,” in Proceedings - 2017 IEEE 31st International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2017, 2017, pp. 1068–1077.
[22] “Embedded Microprocessor Benchmark Consortium homepage.” [Online]. Available: https://www.eembc.org. [Accessed: 16-Aug-2018].
[23] M. R. Guthaus, J. S. Ringenberg, D. Ernst, T. M. Austin, T. Mudge, and R. B. Brown, “MiBench: A free, commercially representative embedded benchmark suite,” 2001 IEEE Int. Work. Workload Charact. WWC 2001, pp. 3–14, 2001.
[24] “MiBench homepage.” [Online]. Available: http://vhosts.eecs.umich.edu/mibench/. [Accessed: 16-Aug-2018].
[25] “Pardus homepage.” [Online]. Available: https://www.pardus.org.tr. [Accessed: 25-Aug2018].