The DTEVV Workshop will be held February 4, 2026 at the USMSM SMART Building.
| 8:00 a.m. | Registration (Coffee and light refreshments will be available) | ----- |
| 8:30 a.m. | Welcome | Reza Ghodssi Executive Director of Research and Innovation Distinguished University Professor University of Maryland MATRIX Lab |
| 8:35 a.m. | Opening Remarks | Samuel Graham, Jr. Dean, A. James Clark School of Engineering University of Maryland |
| 8:45 a.m. | Workshop Overview | Donald "Bucket" Costello III Director of Test and Evaluation of Autonomous Systems University of Maryland MATRIX Lab |
| 9:00 a.m. |
AM Keynote: From "Cool" to Confident: The Unavoidable Journey of Robot Testing and Evaluation Robot performance evaluation is often treated as an afterthought when we are heads down, working on something cool, and rightly so. Methodical performance evaluation can sometimes slow down this process of discovery. Nevertheless, we eventually have to test (formally or informally) whether the thing we conjured up can accomplish the task it was created to complete. This talk will present a brief history of the development of performance metrics, test methods, artifacts, and standards for robots and autonomous systems (and their related technologies) at the National Institute of Standards and Technology (NIST), with examples of and lessons learned from both successes and failures. Full abstract |
Kamel "Kam" Saidi Program Manager and Group Leader, Intelligent Systems Division (ISD) National Institute of Standards and Technology (NIST) |
| 9:45 a.m. | Break | ----- |
| 10:00 a.m. |
AM Panel: Priority Research Directions for DTEVV The increasing importance of test and evaluation has bred various lines of T&E-driven research, often occurring in parallel with specific technology development. From robotics to autonomous systems, T&E/V&V practices have had to evolve to ensure that they capably and rigorously assess emergent technologies. This panel will dive into some of these T&E research activities and the driving technologies, along with the practical insight that they have gained in their efforts. Panelists will be further engaged on successes (even early ones), best practices, and challenges. |
Moderator Panelists George Hwang Jeremy Marvel Charles Rea |
| 11:00 a.m. | Research Roadmap Development | ----- |
| 12:15 p.m. | Break | ----- |
| 12:30 p.m. |
Lunch and PM Keynote NAWCAD Executive Director Stephen Cricchi will discuss the strategic importance of Developmental Test & Evaluation and Verification & Validation in accelerating the delivery of combat-credible capabilities to the warfighter. Lunch will be provided |
Stephen Cricchi Executive Director Naval Air Warfare Center Aircraft Division (NAWCAD) |
| 1:15 p.m. | DTEVV Education Program Overview | Donald "Bucket" Costello III Director of Test and Evaluation of Autonomous Systems University of Maryland MATRIX Lab |
| 1:30 p.m. |
PM Panel: Immediate DTEVV Workforce Needs T&E, and the overall DTEVV community, is only as effective as their workforce capability. Some personnel enter the workforce with key T&E skills that they built through their undergraduate/graduate education, or through internships. Conversely, some practitioners enter the workforce with adjacent skillsets where they dramatically build their T&E prowess through "on-the-job" training and "getting their hands dirty" in actual T&E efforts. Neither is better than the other - it is often that both provide complementary skills that ultimately boost the finished product, which is the successful development and assessment of an emergent technology. This panel will explore the critical DTEVV skills, either built through formal education or in the workplace, that have shaped the DTEVV community. Likewise, this panel will explore the expected skills that will be needed to develop, implement, and assess the technologies of tomorrow. |
Moderator Panelists Mauricio Castillo-Effen CAPT Daniel "Tonto" Kitts, USN Theresa Shafer Leslie Taylor |
| 2:30 p.m. | Workforce Roadmap Development | ----- |
| 3:45 p.m. | Wrapup | Donald "Bucket" Costello III Director of Test and Evaluation of Autonomous Systems University of Maryland MATRIX Lab |
| 4:00 p.m. | Open House/Networking | ----- |
| 5:30 p.m. |
Evening@SMART: Interdisciplinary Research at the Institute for Systems Research This seminar will focus on outlining the interdisciplinary research programs at the Institute for Systems Research (ISR) at the A. James Clark School of Engineering at the University of Maryland, College Park. Since its inception, ISR has been dedicated to advancing the design and control of complex large-scale engineering systems with the continued mission of driving groundbreaking systems research while fostering an environment where interdisciplinary and multidisciplinary collaborations fuel innovation. In this seminar, in addition to outlining ISR's strong research trusts, several recent collaborative research initiatives, with a focus on AI and Autonomy, will be highlighted. Full abstract |
Alireza Khaligh Interim Director University of Maryland Institute for Systems Research |
Christopher C. Collins
George Hwang
Will Becker
Alireza Khaligh
Mauricio Castillo-Effen
CAPT Daniel "Tonto" Kitts, USN
Wayne Dumais
Jeremy Marvel
Stephen Cricchi
Charles Rea
Reza Ghodssi
Kamel "Kam" Saidi
Samuel Graham, Jr.
Leslie Taylor