New Guidelines Aim to Make Technology Research More Accessible and Easier to Verify
ECE Professor Miriam Leeser and her co-authors examine the evolution of research transparency in their editorial, “Artifact Evaluation in the FPGA Community and in ACM TRETS,” published in ACM Transactions on Reconfigurable Technology and Systems. The piece highlights how the journal is adopting artifact badging to overcome the unique hardware and software hurdles of the FPGA field, ensuring published research is more accessible and reproducible.
Abstract:
Artifact evaluation (AE) is gaining traction across the computer science community as a means of advancing reproducible research and strengthening readers’ confidence in published results. Applying reproducibility to computer systems and architecture research has proven particularly challenging, and the FPGA community faces its own distinct hurdles — namely, the use of non-standard hardware platforms and dependence on specific software tools and versions. In this editorial, we review the history of AE in the FPGA community, compare it to practices in related fields, and discuss challenges and future directions. To date, AE has meaningfully improved the availability and accessibility of artifacts and their documentation, while also increasing readers’ confidence in published findings. Looking ahead, AE is poised to continue growing across the reconfigurable hardware community. Notably, ACM Transactions on Reconfigurable Technology and Systems will now offer artifact evaluation with the opportunity to earn artifact badges for all accepted papers.